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Introduction 


PREFACE 


This book provides a basic, conceptual-level description of engineering management disciplines that 
relate to the development and life cycle management of a system. For the non-engineer it provides an 
overview of how a system is developed. For the engineer and project manager it provides a basic framework 
for planning and assessing system development. 

Information in the book is from various sources, but a good portion is taken from lecture material devel¬ 
oped for the two Systems Planning, Research, Development, and Engineering courses offered by the 
Defense Acquisition University. 

The book is divided into four parts: Introduction; Systems Engineering Process; Systems Analysis and 
Control; and Planning, Organizing, and Managing. The first part introduces the basic concepts that 
govern the systems engineering process and how those concepts fit the Department of Defense acquisition 
process. Chapter 1 establishes the basic concept and introduces terms that will be used throughout the 
book. The second chapter goes through a typical acquisition life cycle showing how systems engineering 
supports acquisition decision making. 

The second part introduces the systems engineering problem-solving process, and discusses in basic 
terms some traditional techniques used in the process. An overview is given, and then the process of 
requirements analysis, functional analysis and allocation, design synthesis, and verification is explained 
in some detail. This paid ends with a discussion of the documentation developed as the finished output of 
the systems engineering process. 

Part three discusses analysis and control tools that provide balance to the process. Key activities (such as 
risk management, configuration management, and trade studies) that support and run parallel to the 
system engineering process are identified and explained. 

Part four discusses issues integral to the conduct of a systems engineering effort, from planning to 
consideration of broader management issues. 

In some chapters supplementary sections provide related material that shows common techniques or 
policy-driven processes. These expand the basic conceptual discussion, but give the student a clearer 
picture of what systems engineering means in a real acquisition environment. 


IV 
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CHAPTER 1 


INTRODUCTION TO 
SYSTEMS ENGINEERING 
MANAGEMENT 


1.1 PURPOSE 

The overall organization of this text is described 
in the Preface. This chapter establishes some of 
the basic premises that are expanded throughout 
the book. Basic terms explained in this chapter are 
the foundation for following definitions. Key sys¬ 
tems engineering ideas and viewpoints are pre¬ 
sented, stalling with a definition of a system. 

1.2 DEFINITIONS 
A System Is ... 

Simply stated, a system is an integrated composite 
of people, products, and processes that provide a 
capability to satisfy a stated need or objective. 

Systems Engineering Is... 

Systems engineering consists of two significant 
disciplines: the technical knowledge domain in 
which the systems engineer operates, and systems 
engineering management. This book focuses on 
the process of systems engineering management. 

Three commonly used definitions of systems 
engineering are provided by the best known tech¬ 
nical standards that apply to this subject. They all 
have a common theme: 

• A logical sequence of activities and decisions 
that transforms an operational need into a de¬ 
scription of system performance parameters and 
a preferred system configuration. (MIL-STD- 


499A, Engineering Management, 1 May 1974. 
Now cancelled.) 

• An interdisciplinary approach that encompasses 
the entire technical effort, and evolves into and 
verifies an integrated and life cycle balanced 
set of system people, products, and process solu¬ 
tions that satisfy customer needs. (EIA Standard 
IS-632, Systems Engineering , December 1994.) 

• An interdisciplinary, collaborative approach that 
derives, evolves, and verifies a life-cycle bal¬ 
anced system solution which satisfies customer 
expectations and meets public acceptability. 
(IEEE P1220, Standard for Application and 
Management of the Systems Engineering 
Process, [Final Draft], 26 September 1994.) 

In summary, systems engineering is an interdisci¬ 
plinary engineering management process that 
evolves and verifies an integrated, life-cycle bal¬ 
anced set of system solutions that satisfy customer 
needs. 

Systems Engineering Management Is... 

As illustrated by Figure 1-1, systems engineering 
management is accomplished by integrating three 
major activities: 

• Development phasing that controls the design 
process and provides baselines that coordinate 
design efforts, 

• A systems engineering process that provides 
a structure for solving design problems and 
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Figure 1-1.Three Activities of Systems Engineering Management 


tracking requirements flow through the design 
effort, and 

• Life cycle integration that involves customers 
in the design process and ensures that the system 
developed is viable throughout its life. 

Each one of these activities is necessary to achieve 
proper management of a development effort. Phas¬ 
ing has two major puiposes: it controls the design 
effort and is the major connection between the tech¬ 
nical management effort and the overall acquisi¬ 
tion effort. It controls the design effort by devel¬ 
oping design baselines that govern each level of 
development. It interfaces with acquisition man¬ 
agement by providing key events in the develop¬ 
ment process, where design viability can be as¬ 
sessed. The viability of the baselines developed is 
a major input for acquisition management Mile¬ 
stone (MS) decisions. As a result, the timing and 
coordination between technical development 
phasing and the acquisition schedule is critical to 
maintain a healthy acquisition program. 


The systems engineering process is the heart of 
systems engineering management. Its puipose is 
to provide a structured but flexible process that 
transforms requirements into specifications, archi¬ 
tectures, and configuration baselines. The disci¬ 
pline of this process provides the control and trace- 
ability to develop solutions that meet customer 
needs. The systems engineering process may be 
repeated one or more times during any phase of 
the development process. 

Life cycle integration is necessary to ensure that 
the design solution is viable throughout the life of 
the system. It includes the planning associated with 
product and process development, as well as the 
integration of multiple functional concerns into the 
design and engineering process. In this manner, 
product cycle-times can be reduced, and the need 
for redesign and rework substantially reduced. 

1.3 DEVELOPMENT PHASING 
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Development usually progresses through distinct 
levels or stages: 
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• Concept level, which produces a system concept 
description (usually described in a concept 
study); 

• System level, which produces a system descrip¬ 
tion in performance requirement terms; and 

• Subsystem/Component level, which produces 
first a set of subsystem and component product 
performance descriptions, then a set of 
corresponding detailed descriptions of the 
products’ characteristics, essential for their 
production. 

The systems engineering process is applied to each 
level of system development, one level at a time, 
to produce these descriptions commonly called 
configuration baselines. This results in a series of 
configuration baselines, one at each development 
level. These baselines become more detailed with 
each level. 

In the Department of Defense (DoD) the configu¬ 
ration baselines are called the functional baseline 
for the system-level description, the allocated 
baseline for the subsystem/ componentpeifoimance 


descriptions, and the product baseline for the sub¬ 
system/component detail descriptions. Figure 1-2 
shows the basic relationships between the baselines. 
The triangles represent baseline control decision 
points, and are usually referred to as technical re¬ 
views or audits. 

Levels of Development Considerations 

Significant development at any given level in the 
system hierarchy should not occur until the con¬ 
figuration baselines at the higher levels are con¬ 
sidered complete, stable, and controlled. Reviews 
and audits are used to ensure that the baselines are 
ready for the next level of development. As will be 
shown in the next chapter, this review and audit 
process also provides the necessary assessment of 
system maturity, which supports the DoD 
Milestone decision process. 

1.4 THE SYSTEMS ENGINEERING 
PROCESS 

The systems engineering process is a top-down 
comprehensive, iterative and recursive problem 
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Figure 1-2. Development Phasing 
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solving process, applied sequentially through all 
stages of development, that is used to: 

• Transform needs and requirements into a set of 
system product and process descriptions (add¬ 
ing value and more detail with each level of 
development), 

• Generate information for decision makers, and 

• Provide input for the next level of development. 

As illustrated by Figure 1-3, the fundamental sys¬ 
tems engineering activities are Requirements 
Analysis, Functional Analysis and Allocation, and 
Design Synthesis—all balanced by techniques and 
tools collectively called System Analysis and Con¬ 
trol. Systems engineering controls are used to Pack 
decisions and requirements, maintain technical 
baselines, manage interfaces, manage risks, track 
cost and schedule, track technical performance, 
verify requirements are met, and review/audit the 
progress. 


During the systems engineering process architec¬ 
tures are generated to better describe and under¬ 
stand the system. The word “architecture” is used 
in various contexts in the general field of engi¬ 
neering. It is used as a general description of how 
the subsystems join together to form the system. It 
can also be a detailed description of an aspect of a 
system: for example, the Operational, System, and 
Technical Architectures used in Command, Con¬ 
trol, Communications, Computers, Intelligence, 
Surveillance, and Reconnaissance (C4ISR), and 
software intensive developments. Flowever, Sys¬ 
tems Engineering Management as developed in 
DoD recognizes three universally usable architec¬ 
tures that describe important aspects of the system: 
functional, physical, and system architectures. This 
book will focus on these architectures as neces¬ 
sary components of the systems engineering 
process. 

The Functional Architecture identifies and struc¬ 
tures the allocated functional and performance 
requirements. The Physical Architecture depicts the 
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system product by showing how it is broken down 
into subsystems and components. The System 
Architecture identifies all the products (including 
enabling products) that are necessary to support 
the system and, by implication, the processes 
necessary for development, production/construc¬ 
tion, deployment, operations, support, disposal, 
training, and verification. 

Life Cycle Integration 

Life cycle integration is achieved through inte¬ 
grated development—that is, concurrent consid¬ 
eration of all life cycle needs during the develop¬ 
ment process. DoD policy requires integrated 
development, called Integrated Product and Prod¬ 
uct Development (IPPD) in DoD, to be practiced 
at all levels in the acquisition chain of command 
as will be explained in the chapter on IPPD. Con¬ 
current consideration of all life cycle needs can be 
greatly enhanced through the use of interdiscipli¬ 
nary teams. These teams are often referred to as 
Integrated Product Teams (IPTs). 

The objective of an Integrated Product Team is to: 

• Produce a design solution that satisfies initially 
defined requirements, and 

• Communicate that design solution clearly, 
effectively, and in a timely manner. 

Multi-functional, integrated teams: 

• Place balanced emphasis on product and process 
development, and 

• Require early involvement of all disciplines 
appropriate to the team task. 

Design-level IPT members are chosen to meet the 
team objectives and generally have distinctive com¬ 
petence in: 

• Technical management (systems engineering), 

• Life cycle functional areas (eight primary 
functions), 


• Technical specialty areas, such as safety, risk 
management, quality, etc., or 

• When appropriate, business areas such as 
finance, cost/budget analysis, and contracting. 

Life Cycle Functions 

Life cycle functions are the characteristic actions 
associated with the system life cycle. As illustrated 
by Figure 1-4, they arc development, production 
and construction, deployment (fielding), opera¬ 
tion, support, disposal, training, and verification. 
These activities cover the “cradle to grave” life 
cycle process and are associated with major func¬ 
tional groups that provide essential support to the 
life cycle process. These key life cycle functions 
are commonly referred to as the eight primary 
functions of systems engineering. 

The customers of the systems engineer perform 
the life-cycle functions. The system user’s needs 
are emphasized because their needs generate the 
requirement for the system, but it must be remem¬ 
bered that all of the life-cycle functional areas 
generate requirements for the systems engineer¬ 
ing process once the user has established the basic 
need. Those that perform the primary functions 
also provide life-cycle representation in design- 
level integrated teams. 

Primary Function Definitions 

Development includes the activities required to 
evolve the system from customer needs to product 
or process solutions. 

Manufacturing/Production/Construction in¬ 
cludes the fabrication of engineering test models 
and “brass boards,” low rate initial production, 
full-rate production of systems and end items, or 
the construction of large or unique systems or sub¬ 
systems. 

Deployment (Fielding) includes the activities nec¬ 
essary to initially deliver, transport, receive, pro¬ 
cess, assemble, install, checkout, Pain, operate, 
house, store, or field the system to achieve full 
operational capability. 
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Figure 1-4. Primary Life Cycle Functions 


Operation is the user function and includes 
activities necessary to satisfy defined operational 
objectives and tasks in peacetime and wartime 
environments. 

Support includes the activities necessary to pro¬ 
vide operations support, maintenance, logistics, 
and material management. 

Disposal includes the activities necessary to ensure 
that the disposal of decommissioned, destroyed, 
or irreparable system components meets all 
applicable regulations and directives. 

Training includes the activities necessary to 
achieve and maintain the knowledge and skill levels 
necessary to efficiently and effectively perform 
operations and support functions. 

Verification includes the activities necessary to 
evaluate progress and effectiveness of evolving 
system products and processes, and to measure 
specification compliance. 


Systems Engineering Considerations 

Systems engineering is a standardized, disciplined 
management process for development of system 
solutions that provides a constant approach to 
system development in an environment of change 
and uncertainty. It also provides for simultaneous 
product and process development, as well as a 
common basis for communication. 

Systems engineering ensures that the correct 
technical tasks get done during development 
through planning, tracking, and coordinating. 
Responsibilities of systems engineers include: 

• Development of a total system design solution 
that balances cost, schedule, performance, and 
risk, 

• Development and tracking of technical 
information needed for decision making, 

• Verification that technical solutions satisfy 
customer requirements, 
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• Development of a system that can be produced 
economically and supported throughout the life 
cycle, 

• Development and monitoring of internal and 
external interface compatibility of the sys¬ 
tem and subsystems using an open systems 
approach, 

• Establishment of baselines and configuration 
control, and 

• Proper focus and structure for system and major 
sub-system level design IPTs. 

1.5 GUIDANCE 

DoD 5000.2-R establishes two fundamental 
requirements for program management: 

• It requires that an Integrated Product and 
Process approach be taken to design wherever 
practicable, and 

• It requires that a disciplined systems engineer¬ 
ing process be used to translate operational 
needs and/or requirements into a system 
solution. 

Tailoring the Process 

System engineering is applied during all acquisi¬ 
tion and support phases for large- and small-scale 
systems, new developments or product improve¬ 
ments, and single and multiple procurements. The 
process must be tailored for different needs and/or 
requirements. Tailoring considerations include 
system size and complexity, level of system 
definition detail, scenarios and missions, con¬ 
straints and requirements, technology base, major 
risk factors, and organizational best practices and 
strengths. 

For example, systems engineering of software 
should follow the basic systems engineering 
approach as presented in this book. However, it 
must be tailored to accommodate the software 
development environment, and the unique progress 


tracking and verification problems software devel¬ 
opment entails. In a like manner, all technology 
domains are expected to bring their own unique 
needs to the process. 

This book provides a conceptual-level description 
of systems engineering management. The specific 
techniques, nomenclature, and recommended 
methods are not meant to be prescriptive. Techni¬ 
cal managers must tailor their systems engineer¬ 
ing planning to meet their particular requirements 
and constraints, environment, technical domain, 
and schedule/budget situation. 

However, the basic time-proven concepts inherent 
in the systems engineering approach must be re¬ 
tained to provide continuity and control. For com¬ 
plex system designs, a full and documented un¬ 
derstanding of what the system must do should 
precede development of component performance 
descriptions, which should precede component 
detail descriptions. Though some parts of the sys¬ 
tem may be dictated as a constraint or interface, in 
general, solving the design problem should start 
with analyzing the requirements and determining 
what the system has to do before physical alterna¬ 
tives are chosen. Configurations must be controlled 
and risk must be managed. 

Tailoring of this process has to be done carefully 
to avoid the introduction of substantial unseen risk 
and uncertainty. Without the control, coordination, 
and traceability of systems engineering, an envi¬ 
ronment of uncertainty results which will lead to 
surprises. Experience has shown that these 
surprises almost invariably lead to significant 
impacts to cost and schedule. Tailored processes 
that reflect the general conceptual approach of this 
book have been developed and adopted by profes¬ 
sional societies, academia, industry associations, 
government agencies, and major companies. 

1.6 SUMMARY POINTS 

• Systems engineering management is a multi¬ 
functional process that integrates life cycle 
functions, the systems engineering problem¬ 
solving process, and progressive baselining. 
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• The systems engineering process is a prob¬ 
lem-solving process that drives the balanced 
development of system products and processes. 

• Integrated Product Teams should apply the sys¬ 
tems engineering process to develop a life cycle 
balanced-design solution. 

• The systems engineering process is applied to 
each level of development, one level at a time. 

• Fundamental systems engineering activities are 
Requirements Analysis, Functional Analysis/ 
Allocation, and Design Synthesis, all of which 
are balanced by System Analysis and Control. 


• Baseline phasing provides for an increasing 
level of descriptive detail of the products and 
processes with each application of the systems 
engineering process. 

• Baselining in a nut shell is a concept descrip¬ 
tion that leads to a system definition which, in 
turn, leads to component definitions, and then 
to component designs, which finally lead to a 
product. 

• The output of each application of the systems 
engineering process is a major input to the next 
process application. 
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CHAPTER 2 


SYSTEMS ENGINEERING 
MANAGEMENT IN 
DOD ACQUISITION 


2.1 INTRODUCTION 

The DoD acquisition process has its foundation in 
federal policy and public law. The development, 
acquisition, and operation of military systems is 
governed by a multitude of public laws, formal 
DoD directives, instructions and manuals, numer¬ 
ous Service and Component regulations, and many 
inter-service and international agreements. 

Managing the development and fielding of mili¬ 
tary systems requires three basic activities: tech¬ 
nical management, business management, and con¬ 
tract management. As described in this book, 
systems engineering management is the technical 
management component of DoD acquisition 
management. 

The acquisition process runs parallel to the require¬ 
ments generation process and the budgeting pro¬ 
cess (Planning, Programming, and Budgeting Sys¬ 
tem.) User requirements tend to be event driven 
by threat. The budgeting process is date driven by 
constraints of the Congressional calendar. Systems 
Engineering Management bridges these processes 
and must resolve the dichotomy of event driven 
needs, event driven technology development, and 
a calendar driven budget. 

Direction and Guidance 

The Office of Management and Budget (OMB) 
provides top-level guidance for planning, budget¬ 
ing, and acquisition in OMB Circular A-11, Part 
3, and the Supplemental Capital Programming 
Guide: Planning, Budgeting, and Acquisition of 
Capital Assets, July 1997. These documents 


establish the broad responsibilities and ground 
rules to be followed in funding and acquiring major 
assets. The departments of the executive branch of 
government are then expected to draft their own 
guidance consistent with the guidelines estab¬ 
lished. The principal guidance for defense system 
acquisitions is the DoD 5000 series of directives 
and regulations. These documents reflect the 
actions required of DoD acquisition managers to: 

• Translate operational needs into stable, 
affordable programs, 

• Acquire quality products, and 

• Organize for efficiency and effectiveness. 

2.2 RECENT CHANGES 

The DoD 5000 series documents were revised in 
2000 to make the process more flexible, enabling 
the delivery of advanced technology to warfighters 
more rapidly and at reduced total ownership cost. 
The new process encourages multiple entry points, 
depending on the maturity of the fundamental tech¬ 
nologies involved, and the use of evolutionary meth¬ 
ods to define and develop systems. This encourages 
a tailored approach to acquisition and engineering 
management, but it does not alter the basic logic 
of the underlying systems engineering process. 


2.3 ACQUISITION LIFE CYCLE 

The revised acquisition process for major defense 
systems is shown in Figure 2-1. The process is 
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Figure 2-1. Revised DoD 5000 Acquisition Process 


defined by a series of phases during which tech¬ 
nology is defined and matured into viable concepts, 
which arc subsequently developed and readied for 
production, after which the systems produced are 
supported in the field. 

The process allows for a given system to enter the 
process at any of the development phases. For ex¬ 
ample, a system using unproven technology would 
enter at the beginning stages of the process and 
would proceed through a lengthy period of tech¬ 
nology maturation, while a system based on ma¬ 
ture and proven technologies might enter directly 
into engineering development or, conceivably, even 
production. The process itself (Figure 2-1) includes 
four phases of development. The first. Concept 
and Technology Development, is intended to ex¬ 
plore alternative concepts based on assessments 
of operational needs, technology readiness, risk, 
and affordability. Entry into this phase does not 
imply that DoD has committed to a new acquisi¬ 
tion program; rather, it is the initiation of a pro¬ 
cess to determine whether or not a need (typically 
described in a Mission Need Statement (MNS)) 
can be met at reasonable levels of technical risk 
and at affordable costs. The decision to enter into 


the Concept and Technology Development phase 
is made formally at the Milestone A forum. 

The Concept and Technology Development 

phase begins with concept exploration. During this 
stage, concept studies are undertaken to define al¬ 
ternative concepts and to provide information about 
capability and risk that would permit an objective 
comparison of competing concepts. A decision 
review is held after completion of the concept ex¬ 
ploration activities. The puipose of this review is 
to determine whether further technology develop¬ 
ment is required, or whether the system is ready to 
enter into system acquisition. If the key technolo¬ 
gies involved are reasonably mature and have al¬ 
ready been demonstrated, the Milestone Decision 
Authority (MDA) may agree to allow the system 
to proceed into system acquisition; if not, the sys¬ 
tem may be directed into a component advanced 
development stage. (See Supplement A to this 
chapter for a definition of Technology Readiness 
levels.) During this stage, system architecture defi¬ 
nition will continue and key technologies will be 
demonstrated in order to ensure that technical and 
cost risks are understood and are at acceptable lev¬ 
els prior to entering acquisition. In any event, the 
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Concept and Technology Development phase ends 
with a defined system architecture supported by 
technologies that are at acceptable levels of matu¬ 
rity to justify entry into system acquisition. 

Formal system acquisition begins with a Milestone 
B decision. The decision is based on an integrated 
assessment of technology maturity, user require¬ 
ments, and funding. A successful Milestone B is 
followed by the System Development and Dem¬ 
onstration phase. This phase could be entered di¬ 
rectly as a result of a technological opportunity 
and urgent user need, as well as having come 
through concept and technology development. The 
System Development and Demonstration phase 
consists of two stages of development, system 
integration and system demonstration. Depending 
upon the maturity level of the system, it could enter 
at either stage, or the stages could be combined. 
This is the phase during which the technologies, 
components and subsystems defined earlier are first 
integrated at the system level, and then demon¬ 
strated and tested. If the system has never been 
integrated into a complete system, it will enter this 
phase at the system integration stage. When sub¬ 
systems have been integrated, prototypes demon¬ 
strated, and risks are considered acceptable, the 
program will normally enter the system demon¬ 
stration stage following an interim review by the 
MDA to ensure readiness. The system demonstra¬ 
tion stage is intended to demonstrate that the system 
has operational utility consistent with the opera¬ 
tional requirements. Engineering demonstration 
models are developed and system level develop¬ 
ment testing and operational assessments are per¬ 
formed to ensure that the system performs as 
required. These demonstrations are to be conducted 
in environments that represent the eventual opera¬ 
tional environments intended. Once a system has 
been demonstrated in an operationally relevant 
environment, it may enter the Production and 
Deployment phase. 

The Production and Deployment phase consists 
of two stages: production readiness and low rate 
initial production (LRIP), and rate production 
and deployment. The decision forum for entry into 
this phase is the Milestone C event. Again, the 
fundamental issue as to where a program enters 


the process is a function of technology maturity, 
so the possibility exists that a system could enter 
directly into this phase if it were sufficiently ma¬ 
ture, for example, a commercial product to be pro¬ 
duced for defense applications. Flowever the entry 
is made—directly or through the maturation pro¬ 
cess described, the production readiness and LRIP 
stage is where initial operational test, live fire test, 
and low rate initial production are conducted. Upon 
completion of the LRIP stage and following a 
favorable Beyond LRIP test report, the system enters 
the rate production and deployment stage during 
which the item is produced and deployed to the 
user. As the system is produced and deployed, the 
final phase. Sustainment and Disposal, begins. 

The last, and longest, phase is the Sustainment 
and Disposal phase of the program. During this 
phase all necessary activities are accomplished to 
maintain and sustain the system in the field in the 
most cost-effective manner possible. The scope of 
activities is broad and includes everything from 
maintenance and supply to safety, health, and en¬ 
vironmental management. This period may also 
include transition from contractor to organic sup¬ 
port, if appropriate. During this phase, modifica¬ 
tions and product improvements are usually imple¬ 
mented to update and maintain the required levels 
of operational capability as technologies and threat 
systems evolve. At the end of the system service 
life it is disposed of in accordance with applicable 
classified and environmental laws, regulations, and 
directives. Disposal activities also include recy¬ 
cling, material recovery, salvage of reutilization, 
and disposal of by-products from development and 
production. 

The key to this model of the acquisition process is 
that programs have the flexibility to enter at any 
of the first three phases described. The decision as 
to where the program should enter the process is 
primarily a function of user needs and technology 
maturity. The MDA makes the decision for the 
program in question. Program managers are 
encouraged to work with their users to develop evo¬ 
lutionary acquisition strategies that will permit 
deliveries of usable capabilities in as short a time- 
frame as possible, with improvements and en¬ 
hancements added as needed through continuing 
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definition of requirements and development activi¬ 
ties to support the evolving needs. 

2.4 SYSTEMS ENGINEERING 
IN ACQUISITION 

As required by DoD 5000.2-R. the systems 
engineering process shall: 

1. Transform operational needs and requirements 
into an integrated system design solution 
through concurrent consideration of all life- 
cycle needs (i.e., development, manufacturing, 
test and evaluation, verification, deployment, 
operations, support, training and disposal). 

2. Ensure the compatibility, interoperability and 
integration of all functional and physical inter¬ 
faces and ensure that system definition and 
design reflect the requirements for all system 
elements: hardware, software, facilities, people, 
and data; and 


3. Characterize and manage technical risks. 

4. Apply scientific and engineering principles to 
identify security vulnerabilities and to minimize 
or contain associated information assurance and 
force protection risks. 

These objectives are accomplished with use of the 
management concepts and techniques described in 
the chapters which follow in this book. The appli¬ 
cation of systems engineering management coin¬ 
cides with acquisition phasing. In order to support 
milestone decisions, major technical reviews are 
conducted to evaluate system design maturity. 

Concept and Technology Development 

The Concept and Technology Development phase 
consists of two pre-acquisition stages of develop¬ 
ment. The first. Concept Exploration, is repre¬ 
sented in Figure 2-2. The exploration of concepts 
is usually accomplished through multiple short¬ 
term studies. Development of these studies is 


MNS 



Technology 

Opportunity 

Assessments 


Analysis of Alternatives 
Operational Analysis 
R&D Activities 

Technology Opportunity 
Assessments and Analysis 

Market Research 

Business Process 
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ORD Development 
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Alternative Concepts Defined 
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Key Cost, Schedule, Performance 
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Figure 2-2. Concept and Technology Development (Concept Exploration Stage) 
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expected to employ various techniques including 
the systems engineering process that translates 
inputs into viable concept architectures whose 
functionality can be traced to the requirements. In 
addition, market surveys, Business Process Reengi¬ 
neering activities, operational analysis, and trade 
studies support the process. 

The primary inputs to these activities include 
requirements, in form of the MNS, assessments of 
technology opportunities and status, and the out¬ 
puts from any efforts undertaken to explore poten¬ 
tial solutions. When the contractor studies are 
complete, a specific concept to be pursued is 
selected based on a integrated assessment of tech¬ 
nical performance; technical, schedule and cost 
risk; as well as other relevant factors. A decision 
review is then held to evaluate the concept recom¬ 
mended and the state of technology upon which 
the concept depends. The MDA then makes a 
decision as to whether the concept development 
work needs to be extended or redirected, or whether 
the technology maturity is such that the program 
can proceed directly to either Mile-stone B (System 
Development and Demonstration) or Milestone C 
(Production and Deployment). 


If the details of the concept require definition, 
i.e., the system has yet to be designed and demon¬ 
strated previously, or the system appears to be 
based on technologies that hold significant risk, 
then it is likely that the system will proceed to the 
second stage of the Concept and Technology 
Development phase. This stage. Component 
Advanced Development, is represented in Figure 
2-3. This is also a pre-acquisition stage of devel¬ 
opment and is usually characterized by extensive 
involvement of the DoD Science and Technology 
(S&T) community. The fundamental objectives of 
this stage of development are to define a system- 
level architecture and to accomplish risk-reduction 
activities as required to establish confidence that 
the building blocks of the system are sufficiently 
well-defined, tested and demonstrated to provide 
confidence that, when integrated into higher level 
assemblies and subsystems, they will perform 
reliably. 

Development of a system-level architecture entails 
continuing refinement of system level requirements 
based on comparative analyses of the system con¬ 
cepts under consideration. It also requires that 
consideration be given to the role that the system 
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ORD Development 
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Figure 2-3. Concept and Technology Development 
(Component Advanced Development Stage) 
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will play in the system of systems of which it will 
be a part. System level interfaces must be estab¬ 
lished. Communications and interoperability re¬ 
quirements must be established, data flows defined, 
and operational concepts refined. Top level plan¬ 
ning should also address the strategics that will be 
employed to maintain the supportability and 
affordability of the system over its life cycle 
including the use of common interface standards 
and open systems architectures. Important design 
requirements such as interoperability, open sys¬ 
tems, and the use of commercial components 
should also be addressed during this stage of the 
program. 

Risk reduction activities such as modeling and 
simulation, component testing, bench testing, and 
man-in-the-loop testing are emphasized as deci¬ 
sions are made regarding the various technologies 
that must be integrated to form the system. The 
primary focus at this stage is to ensure that the key 
technologies that represent the system components 
(assemblies and sub-systems) are well understood 


and are mature enough to justify their use in a sys¬ 
tem design and development effort. The next stage 
of the life cycle involves engineering development, 
so research and development (R&D) activities 
conducted within the science and technology 
appropriations should be completed during this 
stage. 

System Development and Demonstration 

The decision forum for entry into the System 
Development and Demonstration (SD&D) phase 
is the Milestone B event. Entry into this phase rep¬ 
resents program initiation , the formal beginning 
of a system acquisition effort. This is the govern¬ 
ment commitment to pursue the program. Entry 
requires mature technology, validated require¬ 
ments, and funding. At this point, the program re¬ 
quirement must be defined by an Operational Re¬ 
quirements Document (ORD). This phase consists 
of two primary stages, system integration (Figure 
2-4) and system demonstration (Figure 2-5). 


Approved Functional Baseline 



Draft Allocated Baseline 



Preliminary and 
Detail Design 
Efforts 


Requirements Tech 

Review Review 
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Demonstration 

System Definition Effort 
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Preliminary Design Effort 


Figure 2-4. System Development and Demonstration 
(System Integration Stage) 
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Figure 2-5. System Development and Demonstration 
(System Demonstration Stage) 


There is no hard and fast guidance that stipulates 
precisely how the systems engineering process is 
to intersect with the DoD acquisition process. 
There are no specified technical events, e.g., DoD 
designated technical reviews, that are to be accom¬ 
plished during identified stages of the SD&D 
phase. However, the results of a SD&D phase 
should support a production go-ahead decision at 
Milestone C. That being the case, the process 
described below reflects a configuration control 
approach that includes a system level design (func¬ 
tional baseline), final preliminary designs (allo¬ 
cated baselines), and detail designs (initial prod¬ 
uct baselines). Along with their associated docu¬ 
mentation, they represent the systems engineering 
products developed during SD&D that are most 
likely needed to appropriately support Milestone C. 

System Integration is that stage of SD&D that 
applies to systems that have yet to achieve system 
level design maturity as demonstrated by the inte¬ 
gration of components at the system level in rel¬ 
evant environments. For an unprecedented system 


(one not previously defined and developed), this 
stage will continue the work begun in the compo¬ 
nent advanced development stage, but the flavor 
of the effort now becomes oriented to engineering 
development, rather than the research-oriented 
efforts that preceded this stage. A formal ORD, 
technology assessments, and a high-level system 
architecture have been established. (These will 
form major inputs to the systems engineering 
process.) The engineering focus becomes estab¬ 
lishment and agreement on system level technical 
requirements stated such that designs based on 
those technical requirements will meet the intent 
of the operational requirements. The system level 
technical requirements are stabilized and docu¬ 
mented in an approved system level requirements 
specification. In addition, the system-level require¬ 
ments baseline (functional baseline) is established. 
This baseline is verified by development and 
demonstration of prototypes that show that key 
technologies can be integrated and that associated 
risks are sufficiently low to justify developing the 
system. 
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Program initiation signals the transition from an 
S&T focus to management by the program office. 
The R&D community, the users, and the program 
office may have all been involved in defining the 
concepts and technologies that will be key to the 
system development. It is appropriate at this point, 
therefore, to conduct a thorough requirements analy¬ 
sis and review to ensure that the user, the contrac¬ 
tor, and the program office all hold a common view 
of the requirements and to preserve the lessons 
learned through the R&D efforts conducted in the 
earlier phase. The risk at this point can be high, 
because misunderstandings and errors regarding 
system-level requirements will flow down to sub¬ 
sequent designs and can eventually result in over¬ 
runs and even program failure. The contractor will 
normally use the occasion of the system require¬ 
ments review early in this stage to set the func¬ 
tional baseline that will govern the flow-down of 
requirements to lower level items as preliminary 
designs are elaborated. 

The Interim Progress Review held between Sys¬ 
tem Integration and System Demonstration has no 
established agenda. The agenda is defined by the 
MDA and can be flexible in its timing and con¬ 
tent. Because of the flexibility built into the 
acquisition process, not all programs will conform 
to the model presented here. Programs may find 
themselves in various stages of preliminary design 
and detailed design as the program passes from 
one stage of the SD&D phase to the succeeding 
stage. With these caveats, System Demonstration 
(Figure 2-5) is the stage of the SD&D phase dur¬ 
ing which preliminary and detailed designs are 
elaborated, engineering demonstration models are 
fabricated, and the system is demonstrated in 
operationally relevant environments. 

System level requirements are flowed down to the 
lower level items in the architecture and require¬ 
ments are documented in the item performance 
specifications, which represent the preliminary 
design requirements for those items. The item per¬ 
formance specifications and supporting documen¬ 
tation, when finalized, together form the allocated 
baseline for the system. Design then proceeds 
toward the elaboration of a detailed design for 


the product or system. The product baseline is 
drafted as the design is elaborated. This physical 
description of the system may change as a result 
of testing that will follow, but it forms the basis 
for initial fabrication and demonstration of these 
items. If the system has been previously designed 
and fabricated, then, clearly, this process would 
be curtailed to take advantage of work already 
completed. 

Following the elaboration of the detailed design, 
components and subsystems arc fabricated, inte¬ 
grated, and tested in a bottom-up approach until 
system level engineering demonstration models arc 
developed. These demonstration models are not, 
as a rule, production representative systems. 
Rather, they are system demonstration models, or 
integrated commercial items, that serve the pur¬ 
pose of enabling the developer to accomplish 
development testing on the integrated system. 
These models are often configured specifically to 
enable testing of critical elements of the system, 
for example, in the case of an aircraft development, 
there may be separate engineering demonstration 
models developed specifically to test the integrated 
avionics subsystems, while others demonstrate the 
flying qualities and flight controls subsystems. 

For purposes of making decisions relative to 
progress through the acquisition process, these 
system-level demonstrations arc not intended to 
be restricted to laboratory test and demonstrations. 
They are expected to include rigorous demonstra¬ 
tions that the integrated system is capable of per¬ 
forming operationally useful tasks under conditions 
that, while not necessarily equal to the rigor of 
formal operational testing, represent the eventual 
environment in which the system must perform. 
The result of these demonstrations provide the 
confidence required to convince the decision¬ 
maker (MDA) that the system is ready to enter the 
production phase of the life cycle. This implies 
that the system has demonstrated not only that 
technical performance is adequate, but also that 
the affordability, supportability, and producibility 
risks are sufficiently low to justify a production 
decision. 
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Production Readiness and LRIP Rate Production and Deployment 


Establish Manufacturing Capability 
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Figure 2-6. Production and Deployment 


Production and Deployment 

Milestone C is the decision forum for entry into 
the Production and Deployment phase of the 
program. Like other phases, this phase is also 
divided into stages of development. Production 
Readiness and LRIP is the first of these. At this 
point, system-level demonstrations have been 
accomplished and the product baseline is defined 
(although it will be refined as a result of the activi¬ 
ties undertaken during this phase). The effort is 
now directed toward development of the manufac¬ 
turing capability that will produce the product or 
system under development. When a manufactur¬ 
ing capability is established, a LRIP effort begins. 

The development of a LRIP manufacturing capa¬ 
bility has multiple purposes. The items produced 
are used to proof and refine the production line 
itself, items produced on this line are used for Ini¬ 
tial Operational Test and Evaluation (IOT&E) and 
Live Fire Test and Evaluation (LFT&E), and this 


is also the means by which manufacturing rates 
are ramped upward to the rates intended when 
manufacturing is fully underway. 

Following the completion of formal testing, the 
submission of required Beyond-LRIP and Live Fire 
Test reports, and a full-rate production decision 
by the MDA, the system enters the Rate Production 
and Deployment stage. After the decision to go to 
full-rate production, the systems engineering 
process is used to refine the design to incorporate 
findings of the independent operational testing, 
direction from the MDA, and feedback from 
deployment activities. Once configuration changes 
have been made and incorporated into production, 
and the configuration and production is consid¬ 
ered stable, Follow-on Operational Test and Evalu¬ 
ation (FOT&E), if required, is typically performed 
on the stable production system. Test results are 
used to further refine the production configuration. 
Once this has been accomplished and production 
again becomes stable, detailed audits are held to 
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confirm that the Product Baseline documentation 
correctly describes the system being produced. 
The Product Baseline is then put under formal 
configuration control. 

As the system is produced, individual items are 
delivered to the field units that will actually em¬ 
ploy and use them in their military missions. Care¬ 
ful coordination and planning is essential to make 
the deployment as smooth as possible. Integrated 
planning is absolutely critical to ensure that the 
training, equipment, and facilities that will be re¬ 
quired to support the system, once deployed, are 
in place as the system is delivered. The systems 
engineering function during this activity is focused 
on the integration of the functional specialties to 
make certain that no critical omission has been 
made that will render the system less effective than 
it might otherwise be. Achieving the user’s required 
initial operational capability (IOC) schedule de¬ 
mands careful attention to the details of the transi¬ 
tion at this point. Furthermore, as the system is 
delivered and operational capability achieved, the 


system transitions to the Sustainment and Disposal 
phase of the system life cycle—the longest and 
most expensive of all phases. 

Sustainment and Disposal 

There is no separate milestone decision required 
for a program to enter this phase of the system life 
cycle. The requirement for the Sustainment phase 
is implicit in the decision to produce and deploy 
the system. This phase overlaps the Production 
phase. Systems Engineering activities in the 
Sustainment phase are focused on maintaining 
the system's performance capability relative to 
the threat the system faces. If the military threat 
changes or a technology opportunity emerges, then 
the system may require modification. These 
modifications must be approved at an appropriate 
level for the particular change being considered. 
The change then drives the initiation of new sys¬ 
tems engineering processes, starting the cycle (or 
parts of it) all over again. 


Sustainment Disposal 



Figure 2-7. Sustainment and Disposal 
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Also, in an evolutionary development environment, 
there will be a continuing effort to develop and 
refine additional operational requirements based 
on the experience of the user with the portion of 
the system already delivered. As new requirements 
are generated, a new development cycle begins, 
with technology demonstrations, risk reduction, 
system demonstrations and testing—the same cycle 
just described—all tailored to the specific needs 
and demands of the technology to be added to the 
core system already delivered. 

The final activity in the system life cycle is Dis¬ 
posal. System engineers plan for and conduct sys¬ 
tem disposal throughout the life cycle beginning 
with concept development. System components 
can require disposal because of decommissioning, 
their destruction, or irreparable damage. In addi¬ 
tion, processes and material used for development, 
production, operation, or maintenance can raise 
disposal issues throughout the life cycle. Disposal 
must be done in accordance with applicable laws, 
regulations, and directives that are continually 
changing, usually to require more severe con¬ 
straints. They mostly relate to security and environ¬ 
ment issues that include recycling, material recov¬ 
ery, salvage, and disposal of by-products from 
development and production. 

Every Development is Different 

The process described above is intended to be very 
flexible in application. There is no “typical” sys¬ 
tem acquisition. The process is therefore defined 
to accommodate a wide range of possibilities, from 
systems that have been proven in commercial 
applications and are being purchased for military 
use, to systems that are designed and developed 
essentially from scratch. The path that the system 
development takes through the process will depend 
primarily on the level of maturity of the technol¬ 
ogy employed. As explained in the preceding dis¬ 
cussion, if the system design will rely significantly 
on the use of proven or commercial items, then 
process can be adjusted to allow the system to skip 
phases, or move quickly from stage to stage within 
phases. If the type of system is well understood 
within the applicable technical domains, or it is an 
advanced version of a current well understood 


system, then the program definition and risk 
reduction efforts could be adjusted appropriately. 

It is the role of the system engineer to advise the 
program manager of the recommended path that 
the development should take, outlining the reasons 
for that recommendation. The decision as to the 
appropriate path through the process is actually 
made by the MDA, normally based on the recom¬ 
mendation of the program manager. The process 
must be tailored to the specific development, both 
because it is good engineering and because it is 
DoD policy as part of the Acquisition Reform ini¬ 
tiative. But tailoring must done with the intent of 
preserving the requirements traceability, baseline 
control, lifecycle focus, maturity tracking, and 
integration inherent in the systems engineering 
approach. The validity of tailoring the process 
should always be a risk management issue. Acqui¬ 
sition Reform issues will be addressed again in Paid 
IV of this text. 


2.5 SUMMARY POINTS 

• The development, acquisition, and operation of 
military systems is governed by a multitude of 
public laws, formal DoD directives, instructions 
and manuals, numerous Service and Compo¬ 
nent regulations, and many inter-service and 
international agreements. 

• The system acquisition life cycle process is a 
model used to guide the program manager through 
the process of maturing technology based sys¬ 
tems and readying them for production and 
deployment to military users. 

• The acquisition process model is intended to 
be flexible and to accommodate systems and 
technologies of varying maturities. Systems 
dependent on immature technologies will take 
longer to develop and produce, while those that 
employ mature technologies can proceed 
through the process relatively quickly. 

• The system engineering effort is integrated into 
the systems acquisition process such that the 
activities associated with systems engineering 
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(development of documentation, technical re¬ 
views, configuration management, etc.) support 
and strengthen the acquisition process. The 
challenge for the engineering manager is to 
ensure that engineering activities are conducted 


at appropriate points in the process to ensure 
that the system has, in fact, achieved the levels 
of maturity expected prior to progressing into 
succeeding phases. 
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SUPPLEMENT 2-A 

TECHNOLOGY 
READINESS LEVELS 


Technology Readiness Level 

Description 

1. Basic principles observed 
and reported. 

Lowest level of technology readiness. Scientific research begins 
to be translated into technology’s basic properties. 

2. Technology concept and/or 
application formulated. 

Invention begins. Once basic principles are observed, practical 
applications can be invented. The application is speculative and 
there is no proof or detailed analysis to support the assumption. 
Examples are still limited to paper studies. 

3. Analytical and experimental 
critical function and/or char¬ 
acteristic proof of concept. 

Active R&D is initiated. This includes analytical studies and 
laboratory studies to physically validate analytical predictions 
of separate elements of the technology. Examples include 
components that are not yet integrated or representative. 

4. Component and/or bread¬ 
board validation in labora¬ 
tory environment. 

Basic technological components are integrated to establish that 
the pieces will work together. This is relatively “low fidelity” 
compared to the eventual system. Examples include integration 
of “ad hoc” hardware in a laboratory. 

5. Component and/or bread¬ 
board validation in relevant 

environment. 

Fidelity of breadboard technology increases significantly. The 
basic technological components are integrated with reasonably 
realistic supporting elements so that the technology can be 
tested in simulated environment. Examples include “high 
fidelity” laboratory integration of components. 

6. System/subsystem model or 
prototype demonstration in a 
relevant environment. 

Representative model or prototype system, which is well beyond 
the breadboard tested for level 5, is tested in a relevant environ¬ 
ment. Represents a major step up in a technology’s demon¬ 
strated readiness. Examples include testing a prototype in a high 
fidelity laboratory environment or in a simulated operational 
environment. 

7. System prototype demon¬ 
stration in an operational 
environment. 

Prototype near or at planned operational system. Represents a 
major step up from level 6, requiring the demonstration of an 
actual system prototype in an operational environment. 

Examples include testing the prototype in a test bed aircraft. 

(continued) 
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Technology Readiness Level 

Description 

8. Actual system completed and 
qualified through test and 
demonstration. 

Technology has been proven to work in its final form and under 
expected conditions. In almost all cases, this level represents the 
end of true system development. Examples include develop¬ 
mental test and evaluation of the system in its intended weapon 
system to determine if it meets design specifications. 

9. Actual system proven 
through successful mission 
operations. 

Actual application of the technology in its final form and under 
mission conditions, such as those encountered in operational 
test and evaluation. Examples include using the system under 
operational mission conditions. 
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SUPPLEMENT 2-B 

EVOLUTIONARY ACQUISITION 
CONSIDERATIONS 


The evolutionary approach to defense acquisition 
is the simple recognition that systems evolve as a 
result of changing user needs, technological 
opportunities, and knowledge gained in operation. 
Evolutionary Acquisition is not new to military 
systems. No naval ship in a class is the same; air¬ 
craft and vehicles have block changes designed to 
improve the design; valiants of systems perform 
different missions; satellites have evolutionary 
improvements between the first and last launched; 
and due to fast evolving technology, computer 
resources and software systems are in constant 
evolution. 


As shown by Figure 2-8, evolutionary acquisition 
starts with the development and delivery of a core 
capability. As knowledge is gained through sys¬ 
tem use and as technology changes, the system is 
evolved to a more useful or effective product. At 
the beginning of an evolutionary acquisition the 
ultimate user need is understood in general terms, 
but a core need that has immediate utility can be 
well-defined. Because future events will affect the 
eventual form of the product, the requirements can 
not be fully defined at the program initiation. How¬ 
ever, the evolutionary development must be accom¬ 
plished in a management system that demands 



Figure 2-8. Evolutionary Acquisition 
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requirements validation, fully funded budgets, and 
rigorous review. In addition, the systems engineer¬ 
ing function remains responsible for controlling 
requirements traceability and configuration con¬ 
trol in the absence of complete definition of all 
requirements or final configurations. These con¬ 
straints and concerns require the evolutionary 
approach be accomplished in a manner such the 
various concerns of users, developers, and man¬ 
agers are adequately addressed, while the risks 
associated with these issues are mitigated. 

Acquisition Managment 

Acquisition management requirements established 
in the DoD 5000 documents and associated com¬ 
ponent regulations or instructions establish a series 
of program-specific analyses, reports, and decision 
documents that support the milestone decision pro¬ 
cess. In addition, prior to decision points in the 
acquisition process, substantial coordination is re¬ 
quired with an array of stakeholders. This process 
is resource consuming but necessary to establish 
the program’s validity in the eyes of those respon¬ 
sible to approve the public resources committed 
to the program. 

Evolutionary acquisition, by its nature, represents 
an “acquisition within an acquisition.” On one 
level, the engineering manager is confronted with 
the management and control of the system as it 
progresses to its eventual final configuration, and, 
on another level, there is the management and con¬ 
trol of the modifications, or blocks, that are suc¬ 
cessively integrated into the system as they are 
developed. The system has associated require¬ 
ments, baselines, reviews—the normal elements 
of a system acquisition; however, each block also 
has specified requirements, configuration, and 
management activities. The challenge for techni¬ 
cal management then becomes to ensure that good 
technical management principles are applied to the 
development of each block, while simultaneously 
ensuring that the definition and control of require¬ 
ments and baselines at the system level include 
and accommodate the evolving architecture. 


System Engineering Concerns 

Evolutionary acquisition will require incremental 
and parallel development activities. These activi¬ 
ties are developing evolutionary designs that 
represent a modification as well as an evolved 
system. The evolutionary upgrade is developed as 
a modification, but the new evolved system must 
be evaluated and verified as a system with new, 
evolved requirements. This implies that, though 
we can enter the acquisition process at any point, 
the basic baselining process required by systems 
engineering must somehow be satisfied for each 
block upgrade to assure requirements traceability 
and configuration control. 

As shown by Figure 2-9, incremental delivery of 
capability can be the result of an evolutionary block 
upgrade or be an incremental release of capa¬ 
bility within the approved program (or current 
evolutionary block) baseline. System engineering 
is concerned with both. There is no check list ap¬ 
proach to structure these relationships, but the fol¬ 
lowing is presented to provide some general guid¬ 
ance in a difficult and complex area of acquisition 
management planning and implementation. 

Evolutionary upgrades may be based on known 
operational requirements where delivery of the 
capability is incremental due to immediate opera¬ 
tional need, continuing refinement of the product 
baseline prior to full operational capability, and 
pre-planned parallel developments. If the modifi¬ 
cation is only at the allocated or product baseline, 
and the program’s approved performance, cost, and 
schedule is not impacted, then the system would 
not necessarily require the management approvals 
and milestones normal to the acquisition process. 

In all cases, the key to maintaining a proper sys¬ 
tems engineering effort is to assure that architec¬ 
tures and configuration baselines used for evolu¬ 
tionary development can be upgraded with mini¬ 
mal impact to documented and demonstrated con¬ 
figurations. The risk associated with this issue can 
be significantly reduced through program planning 
that addresses optimization of the acquisition 
baseline and control of the evolving configuration. 
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A B C 




Figure 2-9. Incremental Release Within Evolutionary Blocks 


Planning 

Evolutionary acquisition program planning must 

clearly define how the core and evolutionary blocks 

will be structured, including: 

1. A clear description of an operationally suitable 
core system including identification of sub¬ 
systems and components most likely to evolve. 

2. Establishment of a process for obtaining, evalu¬ 
ating and integrating operational feedback, 
technology advancements, and emerging 
commercial products. 

3. Planning for evolutionary block upgrade evalu¬ 
ation, requirements validation, and program 
initiation. 

4. Description of the management approach for 
evolutionary upgrades within a block and the 
constraints and controls associated with 
incremental delivery of capability. 

5. Risk analysis of the developmental approach, 
both technical and managerial. 


Systems engineering planning should emphasize: 

1. The openness and modularity of the design 
of the core system architecture in order to 
facilitate modification and upgrades, 

2. How baseline documentation is structured to 
improve flexibility for upgrade, 

3. How evolutionary acquisition planning impacts 
baseline development and documentation 
control, 

4. How technical reviews will be structured to best 
support the acquisition decision points, and 

5. How risk management will monitor and con¬ 
trol the management and technical complexity 
introduced by evolutionary development. 

The basic system architecture should be designed 
to accommodate change. Techniques such as open 
architecting, functional partitioning, modular 
design, and open system design (all described later 
in this book) are key to planning for a flexible 
system that can be easily and affordably modified. 
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Notional Example of Evolutionary MAIS Acquisition Relationships 

Characterization 

System Level 

Acquisition 

Program 

Level 

Acquisition 

Documentation 

Required 

Baseline 

CM Authority 

Overall Need 

Major Program 
or 

Business Area 

Capstone or 
Sub-Portfolio 

Capstone 

Acquisition 

Documentaion 

Top Level 
Functional 
Baseline 

PMO 

Core and 
Evolutionary 
Blocks 

Build or Block 
of 

Major Program 

Acquisition 

Program 

Full 

Program 

Documentation 

Cumulative 
Functional and 
Allocated 
Baseline 

PMO with 
Contractor 
Support 

Incremental 
Delivery of 
Capability 

Release or 
Version 
of Block 

Internal to 
Acquisition 
Program 

Separate 
Acquisition 
Documentation 
Not Required 

Product 

Baseline 

Contractor 
(Must Meet 
Allocated 
Basleine) 

Associated 

Product 

Improvements 

Application 

or 

Bridge 

Parallel Product 
Improvement 
(Less than MAIS) 

Component or 
Lower Decision 
Level Acquisition 
Processing 

Functional, 
Allocated, and 
Product Baselines 

PMO/Contractor 


Table 2-1. Evolutionary Acquisition Relationships 


Example 

Table 2-1 illustrates some of the relationships dis¬ 
cussed above as it might apply to a Major Auto¬ 
mated Information System (MAIS) program. Due 
to the nature of complex software development, a 
MAIS acquisition inevitably will be an evolution¬ 
ary acquisition. In the notional MAIS shown in 
the table, management control is primarily defined 
for capstone, program, subsystem or incremental 
delivery, and supporting program levels. The table 
provides relationships showing how key acquisi¬ 
tion and system engineering activities correlate in 
the evolutionary environment. Probably the most 
important lesson of Table 2-1 is that these rela¬ 
tionships are complex and if they are not planned 
for properly, they will present a significant risk to 
the program. 


Summary 

Acquisition oversight is directly related to the 
performance, cost, and schedule defined in the 
acquisition baseline. It establishes the approved 
scope of the developmental effort. Evolutionary 
development that exceeds the boundaries estab¬ 
lished by the acquisition baseline requires a new 
or revised acquisition review process with addi¬ 
tional oversight requirements. The development 
and approval of the ORD and Acquisition Program 
Baseline are key activities that must structure an 
evolutionary process that provides user and over¬ 
sight needs, budgetary control, requirements 
traceability, risk mitigation, and configuration 
management. 
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CHAPTER 3 


SYSTEMS ENGINEERING 
PROCESS OVERVIEW 


3.1 THE PROCESS 

The Systems Engineering Process (SEP) is a 
comprehensive, iterative and recursive problem 
solving process, applied sequentially top-down by 
integrated teams. It transforms needs and require¬ 
ments into a set of system product and process 
descriptions, generate information for decision 
makers, and provides input for the next level of 
development. The process is applied sequentially, 
one level at a time, adding additional detail and 


definition with each level of development. As 
shown by Figure 3-1, the process includes: inputs 
and outputs; requirements analysis; functional 
analysis and allocation; requirements loop; 
synthesis; design loop; verification; and system 
analysis and control. 

Systems Engineering Process Inputs 

Inputs consist primarily of the customer’s needs, 
objectives, requirements and project constraints. 


Process Input 

• Customer Needs/Objectives/ 
Requirements 

- Missions 

- Measures of Effectiveness 

- Environments 

- Constraints 

• Technology Base 

• Output Requirements from Prior 
Development Effort 

• Program Decision Requirements 

• Requirements Applied Through / 
Specifications and Standards / 


System Analysis 
and Control 
. (Balance) > 


Requirements Analysis 

Analyze Missions and Environments 
Identify Functional Requirements 
Define/Refine Performance and Design 
Constraint Requirements 

Requirements Loop 


Functional Analysis/Allocation 

Decompose to Lower-Level Functions 

Allocate Performance and Other Limiting Requirements 

to All Functional Levels 

Define/Refine Functional Interfaces (Internal/External) 
Define/Refine/Integrate Functional Architecture 


Trade-Off Studies 
Effectiveness Analyses 
Risk Management 
Configuration Management 
Interface Management 
Data Management 
Perfromance Measurement 
-SEMS 
-TPM 

- Technical Reviews 


Design Loop 


Synthesis 


Transform Architectures (Functional to Physical) 
Define Alternative System Concepts, Configuration 
Items and System Elements 
Select Preferred Product and Process Solutions 
Define/Refine Physical Interfaces (Internal/External) 


Related Terms: 

Customer = Organizations responsible for Primary Functions 
Primary Functions = Development, Production/Construction, Verification, 
Deployment, Operations, Support, Training, Disposal 
Systems Elements = Hardware, Software, Personnel, Facilities, Data, Material, 
Services, Techniques 


Process Output 

• Development Level Dependent 

- Decision Database 

- System/Configuration Item 
Architecture 

- Specifications and Baselines 



31 






Systems Engineering Fundamentals 


Chapter 3 


Inputs can include, but are not restricted to, mis¬ 
sions, measures of effectiveness, environments, 
available technology base, output requirements 
from prior application of the systems engineering 
process, program decision requirements, and 
requirements based on “corporate knowledge.” 

Requirements Analysis 

The first step of the Systems Engineering Process 
is to analyze the process inputs. Requirements ana¬ 
lysis is used to develop functional and performance 
requirements; that is, customer requirements are 
translated into a set of requirements that define 
what the system must do and how well it must per¬ 
form. The systems engineer must ensure that the 
requirements are understandable, unambiguous, 
comprehensive, complete, and concise. 

Requirements analysis must clarify and define 
functional requirements and design constraints. 
Functional requirements define quantity (how 
many), quality (how good), coverage (how far), 
time lines (when and how long), and availability 
(how often). Design constraints define those fac¬ 
tors that limit design flexibility, such as: environ¬ 
mental conditions or limits; defense against inter¬ 
nal or external threats; and contract, customer or 
regulatory standards. 

Functional Analysis/Allocation 

Functions are analyzed by decomposing higher- 
level functions identified through requirements 
analysis into lower-level functions. The perfor¬ 
mance requirements associated with the higher 
level are allocated to lower functions. The result is 
a description of the product or item in terms of 
what it does logically and in terms of the perfor¬ 
mance required. This description is often called 
the functional architecture of the product or item. 
Functional analysis and allocation allows for a bet¬ 
ter understanding of what the system has to do, in 
what ways it can do it, and to some extent, the 
priorities and conflicts associated with lower-level 
functions. It provides information essential to 
optimizing physical solutions. Key tools in func¬ 
tional analysis and allocation are Functional Flow 


Block Diagrams, Time Fine Analysis, and the 
Requirements Allocation Sheet. 

Requirements Loop 

Performance of the functional analysis and allo¬ 
cation results in a better understanding of the 
requirements and should prompt reconsideration 
of the requirements analysis. Each function iden¬ 
tified should be traceable back to a requirement. 
This iterative process of revisiting requirements 
analysis as a result of functional analysis and 
allocation is referred to as the requirements loop. 

Design Synthesis 

Design synthesis is the process of defining the 
product or item in terms of the physical and soft¬ 
ware elements which together make up and define 
the item. The result is often referred to as the physi¬ 
cal architecture. Each part must meet at least one 
functional requirement, and any part may support 
many functions. The physical architecture is the 
basic structure for generating the specifications and 
baselines. 

Design Loop 

Similar to the requirements loop described above, 
the design loop is the process of revisiting the func¬ 
tional architecture to verify that the physical design 
synthesized can perform the required functions at 
required levels of performance. The design loop 
permits reconsideration of how the system will 
perform its mission, and this helps optimize the 
synthesized design. 

Verification 

For each application of the system engineering 
process, the solution will be compared to the re¬ 
quirements. This part of the process is called the 
verification loop, or more commonly, Verification. 
Each requirement at each level of development 
must be verifiable. Baseline documentation devel¬ 
oped during the systems engineering process must 
establish the method of verification for each 
requirement. 
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Appropriate methods of verification include 
examination, demonstration, analysis (including 
modeling and simulation), and testing. Formal test 
and evaluation (both developmental and opera¬ 
tional) are important contributors to the verification 
of systems. 

Systems Analysis and Control 

Systems Analysis and Control include technical 
management activities required to measure 
progress, evaluate and select alternatives, and docu¬ 
ment data and decisions. These activities apply to 
all steps of the sysems engineering process. 

System analysis activities include trade-off stud¬ 
ies, effectiveness analyses, and design analyses. 
They evaluate alternative approaches to satisfy 
technical requirements and program objectives, and 
provide a rigorous quantitative basis for selecting 
performance, functional, and design requirements. 
Tools used to provide input to analysis activities 
include modeling, simulation, experimentation, 
and test. 

Control activities include risk management, con¬ 
figuration management, data management, and 
performance-based progress measurement includ¬ 
ing event-based scheduling, Technical Perfor¬ 
mance Measurement (TPM), and technical 
reviews. 

The puipose of Systems Analysis and Control is 
to ensure that: 

• Solution alternative decisions are made only 
after evaluating the impact on system effective¬ 
ness, life cycle resources, risk, and customer 
requirements, 

• Technical decisions and specification require¬ 
ments are based on systems engineering 
outputs, 


• Traceability from systems engineering process 
inputs to outputs is maintained, 

• Schedules for development and delivery are 
mutually supportive, 

• Required technical disciplines are integrated 
into the systems engineering effort, 

• Impacts of customer requirements on resulting 
functional and performance requirements are 
examined for validity, consistency, desirability, 
and attainability, and, 

• Product and process design requirements are 
directly traceable to the functional and perfor¬ 
mance requirements they were designed to 
fulfill, and vice versa. 

Systems Engineering Process Output 

Process output is dependent on the level of devel¬ 
opment. It will include the decision database, the 
system or configuration item architecture, and the 
baselines, including specifications, appropriate to 
the phase of development. In general, it is any data 
that describes or controls the product configura¬ 
tion or the processes necessary to develop that 
product. 

3.2 SUMMARY POINTS 

• The system engineering process is the engine 
that drives the balanced development of sys¬ 
tem products and processes applied to each level 
of development, one level at a time. 

• The process provides an increasing level of 
descriptive detail of products and processes with 
each system engineering process application. 
The output of each application is the input to 
the next process application. 
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CHAPTER 4 


REQUIREMENTS 

ANALYSIS 


4.1 SYSTEMS ENGINEERING PROCESS 
INPUTS 

The inputs to the process include the customer’s 
requirements and the project constraints. Require¬ 
ments relate directly to the performance charac¬ 
teristics of the system being designed. They are 
the stated life-cycle customer needs and objectives 
for the system, and they relate to how well the 
system will work in its intended environment. 

Constraints are conditions that exist because of 
limitations imposed by external interfaces, project 
support, technology, or life cycle support systems. 
Constraints bound the development teams’ design 
opportunities. 

Requirements are the primary focus in the systems 
engineering process because the process’s primary 
purpose is to transform the requirements into de¬ 
signs. The process develops these designs within 


the constraints. They eventually must be verified 
to meet both the requirements and constraints. 

Types of Requirements 

Requirements are categorized in several ways. The 
following are common categorizations of require¬ 
ments that relate to technical management: 

Customer Requirements: Statements of fact and 
assumptions that define the expectations of the 
system in terms of mission objectives, environ¬ 
ment, constraints, and measures of effectiveness 
and suitability (MOE/MOS). The customers are 
those that perform the eight primary functions of 
systems engineering (Chapter 1), with special 
emphasis on the operator as the key customer. 
Operational requirements will define the basic need 
and, at a minimum, answer the questions posed in 
Figure 4-1. 


Operational distribution or deployment: Where will the system be used? 

Mission profile or scenario: How will the system accomplish its mission objective? 

Performance and related parameters: What are the critical system parameters to accom¬ 
plish the mission? 

Utilization environments: How are the various system components to be used? 

Effectiveness requirements: How effective or efficient must the system be in performing its 
mission? 

Operational life cycle: How long will the system be in use by the user? 

Environment: What environments will the system be expected to operate in an effective 
manner? 

Figure 4-1. Operational Requirements - Basic Questions 
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Functional Requirements: The necessary task, 
action or activity that must be accomplished. Func¬ 
tional (what has to be done) requirements identified 
in requirements analysis will be used as the top- 
level functions for functional analysis. 

Performance Requirements: The extent to which 
a mission or function must be executed; generally 
measured in terms of quantity, quality, coverage, 
timeliness or readiness. During requirements analy¬ 
sis, performance (how well does it have to be done) 
requirements will be interactively developed across 
all identified functions based on system life cycle 
factors; and characterized in terms of the degree 
of certainty in their estimate, the degree of criti¬ 
cality to system success, and their relationship to 
other requirements. 

Design Requirements: The “build to,” “code to,” 
and “buy to” requirements for products and “how 
to execute” requirements for processes expressed 
in technical data packages and technical manuals. 

Derived Requirements: Requirements that are 
implied or transformed from higher-level require¬ 
ment. For example, a requirement for long range 
or high speed may result in a design requirement 
for low weight. 

Allocated Requirements: A requirement that is 
established by dividing or otherwise allocating a 
high-level requirement into multiple lower-level 
requirements. Example: A 100-pound item that 
consists of two subsystems might result in weight 
requirements of 70 pounds and 30 pounds for the 
two lower-level items. 

Attributes of Good Requirements 

The attributes of good requirements include the 
following: 

• A requirement must be achievable. It must 
reflect need or objective for which a solution is 
technically achievable at costs considered 
affordable. 


• It must be verifiable—that is, not defined by 
words such as excessive, sufficient, resistant, 
etc. The expected performance and functional 
utility must be expressed in a manner that 
allows verification to be objective, preferably 
quantitative. 

• A requirement must be unambiguous. It must 
have but one possible meaning. 

• It must be complete and contain all mission 
profiles, operational and maintenance concepts, 
utilization environments and constraints. All 
information necessary to understand the 
customer’s need must be there. 

• It must be expressed in terms of need, not 
solution; that is, it should address the “why” 
and “what” of the need, not how to do it. 

• It must be consistent with other requirements. 
Conflicts must be resolved up front. 

• It must be appropriate for the level of system 
hierarchy. It should not be too detailed that it 
constrains solutions for the current level of 
design. For example, detailed requirements 
relating to components would not normally be 
in a system-level specification. 

4.2 REQUIREMENTS ANALYSIS 

Requirements analysis involves defining customer 
needs and objectives in the context of planned 
customer use, environments, and identified sys¬ 
tem characteristics to determine requirements 
for system functions. Prior analyses are reviewed 
and updated, refining mission and environment 
definitions to support system definition. 

Requirements analysis is conducted iteratively with 
functional analysis to optimize performance 
requirements for identified functions, and to 
verify that synthesized solutions can satisfy cus¬ 
tomer requirements. The puipose of Requirements 
Analysis is to: 
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• Refine customer objectives and requirements; 

• Define initial performance objectives and refine 
them into requirements; 

• Identify and define constraints that limit 
solutions; and 

• Define functional and performance require¬ 
ments based on customer provided measures 
of effectiveness. 

In general, Requirements Analysis should result 

in a clear understanding of: 

• Functions: What the system has to do, 

• Performance: Flow well the functions have to 
be performed, 

• Interfaces: Environment in which the system 
will perform, and 

• Other requirements and constraints. 

The understandings that come from requirements 

analysis establish the basis for the functional and 

physical designs to follow. Good requirements 


analysis is fundamental to successful design 
definition. 

Inputs 

Typical inputs include customer needs and objec¬ 
tives, missions, MOE/MOS, environments, key 
performance parameters (KPPs), technology base, 
output requirements from prior application of SEP, 
program decision requirements, and suitability 
requirements. (See Figure 4-2 for additional 
considerations.) 

Input requirements must be comprehensive and 
defined for both system products and system pro¬ 
cesses such as development, manufacturing, veri¬ 
fication, deployment, operations, support, training 
and disposal (eight primary functions). 

Role of Integrated Teams 

The operator customers have expertise in the 
operational employment of the product or item 
being developed. The developers (government and 
contractor) are not necessarily competent in the 
operational aspects of the system under develop¬ 
ment. Typically, the operator’s need is neither 
clearly nor completely expressed in a way directly 


• Inputs converted to putputs: 

- Customer requirements 

- Mission and MOEs (MNS, ORD) 

- Maintenance concept and other life-cycle function 
planning 

- SE outputs from prior development efforts 

Controls 


• Controls: 

- Laws and organizational policies and procedures 

1 


- Military specific requirements 

- Utilization environments 

- Tech base and other constraints Transformed 

into Outputs ^ 

• Enablers: 

- Multi-disciplinary product teams 

- Decision and requirements database including 
system/configuration item descriptions from prior 
efforts 

- System analysis and control 

Requirements 

Analysis 

^^^^-Outputs 

t 

Enablers 



Figure 4-2. Inputs to Requirements Analysis 
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usable by developers. It is unlikely that develop¬ 
ers will receive a well-defined problem from which 
they can develop the system specification. Thus, 
teamwork is necessary to understand the 
problem and to analyze the need. It is imperative 
that customers are part of the definition team. 

On the other hand, customers often find it easier 
to describe a system that attempts to solve the prob¬ 
lem rather than to describe the problem itself. 
Although these “solutions” may be workable to 
some extent, the optimum solution is obtained 
through a proper technical development effort 
that properly balances the various customer mis¬ 
sion objectives, functions, MOE/MOS, and con¬ 
straints. An integrated approach to product and 
process development will balance the analysis of 
requirements by providing understanding and 
accommodation among the eight primary functions. 

Requirements Analysis Questions 

Requirements Analysis is a process of inquiry and 
resolution. The following are typical questions that 
can initiate the thought process: 

• What are the reasons behind the system 
development? 

• What are the customer expectations? 

• Who are the users and how do they intend to 
use the product? 

• What do the users expect of the product? 

• What is their level of expertise? 

• With what environmental characteristics must 
the system comply? 

• What are existing and planned interfaces? 

• What functions will the system perform, 
expressed in customer language? 

• What are the constraints (hardware, software, 
economic, procedural) to which the system must 
comply? 


• What will be the final form of the product: such 
as model, prototype, or mass production? 

This list can start the critical, inquisitive outlook 
necessary to analyze requirements, but it is only 
the beginning. A tailored process similar to the 
one at the end of this chapter must be developed 
to produce the necessary requirements analysis 
outputs. 

4.3 REQUIREMENTS ANALYSIS 
OUTPUTS 

The requirements that result from requirements 
analysis are typically expressed from one of three 
perspectives or views. These have been described 
as the Operational, Functional, and Physical views. 
All three are necessary and must be coordinated 
to fully understand the customers’ needs and 
objectives. All three are documented in the decision 
database. 

Operational View 

The Operational View addresses how the system 
will serve its users. It is useful when establishing 
requirements of “how well” and “under what con¬ 
dition.” Operational view information should be 
documented in an operational concept document 
that identifies: 

• Operational need definition, 

• System mission analysis, 

• Operational sequences, 

• Operational environments, 

• Conditions/events to which a system must 
respond, 

• Operational constraints on system, 

• Mission performance requirements, 

• User and maintainer roles (defined by job tasks 
and skill requirements or constraints). 
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• Structure of the organizations that will operate, 
support and maintain the system, and 

• Operational interfaces with other systems. 

Analyzing requirements requires understanding 
the operational and other life cycle needs and 
constraints. 

Functional View 

The Functional View focuses on WFIAT the sys¬ 
tem must do to produce the required operational 
behavior. It includes required inputs, outputs, 
states, and transformation rules. The functional 
requirements, in combination with the physical 
requirements shown below, are the primary sources 
of the requirements that will eventually be reflected 
in the system specification. Functional View 
information includes: 

• System functions, 

• System performance, 

- Qualitative — how well 

- Quantitative — how much, capacity 

- Timeliness — how often 

• Tasks or actions to be performed, 

• Inter-function relationships, 

• Hardware and software functional relationships, 

• Performance constraints, 

• Interface requirements including identification 
of potential open-system opportunities (poten¬ 
tial standards that could promote open systems 
should be identified), 

• Unique hardware or software, and 

• Verification requirements (to include inspection, 
analysis/simulation, demo, and test). 


Physical View 

The Physical View focuses on FIOW the system is 
constructed. It is key to establishing the physical 
interfaces among operators and equipment, and 
technology requirements. Physical View 
information would normally include: 

• Configuration of System: 

- Interface descriptions, 

- Characteristics of information displays and 
operator controls, 

- Relationships of operators to system/ 
physical equipment, and 

- Operator skills and levels required to 
perform assigned functions. 

• Characterization of Users: 

- Flandicaps (special operating environments), 
and 

- Constraints (movement or visual limita¬ 
tions). 

• System Physical Limitations: 

- Physical limitations (capacity, power, size, 
weight), 

- Technology limitations (range, precision, 
data rates, frequency, language), 

- Government Furinished Equipment (GFE), 
Commercial-Off-the-Shelf (COTS), 
Nondevelopmental Item (NDI), reusability 
requirements, and 

- Necessary or directed standards. 

4.4 SUMMARY POINTS 

• An initial statement of a need is seldom defined 
clearly. 

• A significant amount of collaboration between 
various life cycle customers is necessary to 
produce an acceptable requirements document. 

• Requirements are a statement of the problem 
to be solved. Unconstrained and noninte- 
grated requirements are seldom sufficient for 
designing a solution. 
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• Because requirements from different custom- must be accomplished in order to select a bal¬ 
ers will conflict, constraints will limit options, anced set of requirements that provide feasible 

and resources are not unlimited; trade studies solutions to customer needs. 
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SUPPLEMENT 4-A 

A PROCEDURE FOR 
REQUIREMENTS ANALYSIS 


The following section provides a list of tasks that 
represents a plan to analyze requirements. Part of 
this notional process is based on the 15 require¬ 
ments analysis tasks listed in IEEE PI220. This 
industry standard and others should be consulted 
when preparing engineering activities to help 
identify and structure appropriate activities. 

As with all techniques, the student should be care¬ 
ful to tailor; that is, add or subtract, as suits the 
particular' system being developed. Additionally, 
these tasks, though they build on each other, should 
not be considered purely sequential. Every task 
contributes understanding that may cause a need 
to revisit previous task decisions. This is the nature 
of all System Engineering activities. 

Preparation: Establish and 
Maintain Decision Database 

When beginning a systems engineering process, 
be sure that a system is in place to record and man¬ 
age the decision database. The decision database 
is an historical database of technical decisions and 


requirements for future reference. It is the primary 
means for maintaining requirements traceability. 
This database decision management system must 
be developed or the existing system must be 
reviewed and upgraded as necessary to accommo¬ 
date the new stage of product development. A key 
part of this database management system is a 
Requirements Traceability Matrix that maps re¬ 
quirements to subsystems, configuration items, and 
functional areas. 

This must be developed, updated, and reissued on 
a regular basis. All requirements must be recorded. 
Remember: If it is not recorded, it cannot be an 
approved requirement! 

The 15 Tasks of IEEE P1220 

The IEEE Systems Engineering Standard offers a 
process for performing Requirements Analysis that 
comprehensively identifies the important tasks that 
must be performed. These 15 task areas to be ana¬ 
lyzed follow and are shown in Figure 4-3. 


1 . 

Customer expectations 

9. 

Life cycle 

2. 

Project and enterprise constraints 

10. 

Functional requirements 

3. 

External constraints 

11. 

Performance requirements 

4. 

Operational scenarios 

12. 

Modes of operation 

5. 

Measure of effectiveness (MOEs) 

13. 

Technical performance measures 

6. 

System boundaries 

14. 

Physical characteristics 

7. 

Interfaces 

15. 

Human systems integration 

8. 

Utilization environments 




Figure 4-3. IEEE PI 220 Requirements Analysis Task Areas 
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Task 1. Customer Expectations 

Define and quantify customer expectations. They 
may come from any of the eight primary functions, 
operational requirements documents, mission 
needs, technology-based opportunity, direct com¬ 
munications with customer, or requirements from 
a higher system level. The puipose of this task is 
to determine what the customer wants the system 
to accomplish, and how well each function must 
be accomplished. This should include natural and 
induced environments in which the product(s) of 
the system must operate or be used, and constraints 
(e.g. funding, cost, or price objectives, schedule, 
technology, nondevelopmental and reusable items, 
physical characteristics, hours of operation per day, 
on-off sequences, etc.). 

Task 2. Project and Enterprise Constraints 

Identify and define constraints impacting design 
solutions. Project specific constraints can include: 

• Approved specifications and baselines devel¬ 
oped from prior applications of the Systems 
Engineering Process, 

• Costs, 

• Updated technical and project plans, 

• Team assignments and structure, 

• Control mechanisms, and 

• Required metrics for measuring progress. 
Enterprise constraints can include: 

• Management decisions from a preceding 
technical review, 

• Enterprise general specifications, 

• Standards or guidelines, 

• Policies and procedures, 

• Domain technologies, and 


• Physical, financial, and human resource 
allocations to the project. 

Task 3. External Constraints 

Identify and define external constraints impacting 
design solutions or implementation of the Systems 
Engineering Process activities. External constraints 
can include: 

• Public and international laws and regulations, 

• Technology base, 

• Compliance requirements: industry, interna¬ 
tional, and other general specifications, stan¬ 
dards, and guidelines which require compliance 
for legal, interoperability, or other reasons, 

• Threat system capabilities, and 

• Capabilities of interfacing systems. 

Task 4. Operational Scenarios 

Identify and define operational scenarios that scope 
the anticipated uses of system product(s). For each 
operational scenario, define expected: 

• Interactions with the environment and other 
systems, and 

• Physical interconnectivities with interfacing 
systems, platforms, or products. 

Task 5. Measures of Effectiveness and 
Suitability (MOE/MOS) 

Identify and define systems effectiveness measures 
that reflect overall customer expectations and 
satisfaction. MOEs are related to how well the 
system must perform the customer’s mission. Key 
MOEs include mission performance, safety, oper¬ 
ability, reliability, etc. MOSs are related to how 
well the system performs in its intended environ¬ 
ment and includes measures of supportability, 
maintainability, ease of use, etc. 
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Task 6. System Boundaries 

Define system boundaries including: 

• Which system elements are under design con¬ 
trol of the performing activity and which fall 
outside of their control, and 

• The expected interactions among system ele¬ 
ments under design control and external and/or 
higher-level and interacting systems outside the 
system boundary (including open systems 
approaches). 

Task 7. Interfaces 

Define the functional and physical interfaces to 
external or higher-level and interacting systems, 
platforms, and/or products in quantitative terms 
(include open systems approach). Functional and 
physical interfaces would include mechanical, elec¬ 
trical, thermal, data, control, procedural, and other 
interactions. Interfaces may also be considered 
from an internal/external perspective. Internal 
interfaces are those that address elements inside 
the boundaries established for the system ad¬ 
dressed. These interfaces are generally identified 
and controlled by the contractor responsible for 
developing the system. External interfaces, on the 
other hand, are those which involve entity rela¬ 
tionships outside the established boundaries, and 
these are typically defined and controlled by the 
government. 

Task 8. Utilization Environments 

Define the environments for each operational 
scenario. All environmental factors (natural or 
induced) which may impact system performance 
must be identified and defined. Environmental 
factors include: 

• Weather conditions (e.g., rain, snow, sun, wind, 
ice, dust, fog), 

• Temperature ranges, 

• Topologies (e.g., ocean, mountains, deserts, 
plains, vegetation), 


• Biological (e.g., animal, insects, birds, fungi), 

• Time (e.g., dawn, day, night, dusk), and 

• Induced (e.g., vibration, electromagnetic, 
chemical). 

Task 9. Life Cycle Process Concepts 

Analyze the outputs of tasks 1-8 to define key life 
cycle process requirements necessary to develop, 
produce, test, distribute, operate, support, train, and 
dispose of system products under development. 
Use integrated teams representing the eight primary 
functions. Focus should be on the cost drivers and 
higher risk elements that are anticipated to impact 
supportability and affordability over the useful life 
of the system. 

Task 10. Functional Requirements 

Define what the system must accomplish or must 
be able to do. Functions identified through require¬ 
ments analysis will be further decomposed during 
functional analysis and allocation. 

Task 11. Performance Requirements 

Define the performance requirements for each 
higher-level function performed by the system. Pri¬ 
mary focus should be placed on performance re¬ 
quirements that address the MOEs, and other 
KPPs established in test plans or identified as 
interest items by oversight authorities. 

Task 12. Modes of Operation 

Define the various modes of operation for the sys¬ 
tem products under development. Conditions (e.g., 
environmental, configuration, operational, etc.) that 
determine the modes of operation should be 
included in this definition. 

Task 13. Technical Performance Measures 
(TPMs) 

Identify the key indicators of system performance 
that will be tracked during the design process. 
Selection of TPMs should be limited to critical 
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technical thresholds and goals that, if not met, put 
the project at cost, schedule, or performance risk. 
TPMs involve tracking the actual versus planned 
progress of KPPs such that the manager can make 
judgments about technical progress on a by-ex- 
ception basis. To some extent TPM selection is 
phase dependent. They must be reconsidered at 
each systems engineering process step and at the 
beginning of each phase. 

Task 14. Physical Characteristics 

Identify and define required physical characteris¬ 
tics (e.g., color, texture, size, weight, buoyancy) 
for the system products under development. Iden¬ 
tify which physical characteristics are true con¬ 
straints and which can be changed, based on trade 
studies. 

Task 15. Human Factors 

Identify and define human factor considerations 
(e.g., physical space limits, climatic limits, eye 
movement, reach, ergonomics) which will affect 
operation of the system products under develop¬ 
ment. Identify which human systems integration 
are constraints and which can be changed based 
on trade studies. 

Follow-on Tasks 

The follow-on tasks are related to the iterative 
nature of the Systems Engineering Process: 


Integrate Requirements: 

Take an integrated team approach to requirements 
determination so that conflicts among and between 
requirements are resolved in ways that result in 
design requirements that are balanced in terms of 
both risk and affordability. 

Validate Requirements: 

During Functional Analysis and Allocation, vali¬ 
date that the derived functional and performance 
can be traced to the operational requirements. 

Verify Requirements: 

• Coordinate design, manufacturing, deployment 
and test processes, 

• Ensure that requirements are achievable and 
testable, 

• Verify that the design-to-cost goals are 
achievable, and 

• Verify that the functional and physical archi¬ 
tectures defined during Functional Analysis/ 
Allocation and Synthesis meet the integrated 
technical, cost, and schedule requirements 
within acceptable levels of risk. 


44 



CHAPTER 5 


FUNCTIONAL ANALYSIS 
AND ALLOCATION 


5.1 INTRODUCTION 

The puipose of this systems engineering process 
activity is to transform the functional, performance, 
interface and other requirements that were identi¬ 
fied through requirements analysis into a coherent 
description of system functions that can be used 
to guide the Design Synthesis activity that follows. 
The designer will need to know what the system 
must do, how well, and what constraints will limit 
design flexibility. 

This is accomplished by arranging functions in 
logical sequences, decomposing higher-level 
functions into lower-level functions, and allocat¬ 
ing performance from higher- to lower-level func¬ 
tions. The tools used include functional flow block 
diagrams and timeline analysis; and the product is 
a functional architecture, i.e., a description of the 
system—but in terms of functions and performance 
parameters, rather than a physical description. 
Functional Analysis and Allocation facilitates 
traceability from requirements to the solution 
descriptions that are the outcome of Design 
Synthesis. 

Functions are discrete actions (use action verbs) 
necessary to achieve the system’s objectives. These 
functions may be stated explicitly, or they may be 
derived from stated requirements. The functions 
will ultimately be performed or accomplished 
through use of equipment, personnel, facilities, 
software, or a combination. 

5.2 FUNCTIONAL ANALYSIS AND 
ALLOCATION 

Functional and performance requirements at any 
level in the system are developed from higher-level 


requirements. Functional Analysis and Allocation 
is repeated to define successively lower-level func¬ 
tional and performance requirements, thus defin¬ 
ing architectures at ever-increasing levels of detail. 
System requirements are allocated and defined in 
sufficient detail to provide design and verification 
criteria to support the integrated system design. 

This top-down process of translating system- 
level requirements into detailed functional and 
performance design criteria includes: 

• Defining the system in functional terms, then 
decomposing the top-level functions into 
subfunctions. That is, identifying at successively 
lower levels what actions the system has to do, 

• Translating higher-level performance require¬ 
ments into detailed functional and performance 
design criteria or constraints. That is, identi¬ 
fying how well the functions have to be 
performed, 

• Identifying and defining all internal and external 
functional interfaces, 

• Identifying functional groupings to minimize 
and control interfaces (functional partitioning), 

• Determining the functional characteristics of exist¬ 
ing or directed components in the system and in¬ 
corporating them in the analysis and allocation, 

• Examining all life cycle functions, including 
the eight primary functions, as appropriate for 
the specific project, 

• Performing hade studies to determine alterna¬ 
tive functional approaches to meet requirements, 
and 
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• Revisiting the requirements analysis step as 
necessary to resolve functional issues. 

The objective is to identify the functional, per¬ 
formance, and interface design requirements; it 
is not to design a solution...yet! 

Functional Partitioning 

Functional partitioning is the process of grouping 
functions that logically fit with the components 
likely to be used, and to minimize functional in¬ 
terfaces. Partitioning is performed as part of func¬ 
tional decomposition. It identifies logical group¬ 
ings of functions that facilitate the use of modular 
components and open-system designs. Functional 
partitioning is also useful in understanding how 
existing equipment or components (including 
commercial) will function with or within the 
system. 


Requirements Loop 

During the performance of the Functional Analysis 
and Allocation process, it is expected that revisit¬ 
ing the requirements analysis process will be 
necessary. This is caused by the emergence of 
functional issues that will require re-examination 
of the higher-level requirements. Such issues might 
include directed components or standards that 
cause functional conflict, identification of a revised 
approach to functional sequencing, or, most likely, 
a conflict caused by mutually incompatible 
requirements. 

Figure 5-1 gives an overview of the basic param¬ 
eters of Functional Analysis and Allocation. The 
output of the process is the functional architec¬ 
ture. In its most basic form, the functional archi¬ 
tecture is a simple hierarchical decomposition of 
the functions with associated performance require¬ 
ments. As the architecture definition is refined and 
made more specific with the performance of the 


• Outputs: 

- Functional architecture and supporting detail 



• Inputs: 

- Outputs of the Requirements Analysis 



• Enablers: 

- Multi-discipline product teams, decision database; Tools & Models, such as QFD, Functional Flow 
Block Diagrams, IDEF, N2 charts, Requirement Allocation Sheet, Timelines, Data Flow Diagrams, 
State/Mode Diagrams, Behavior Diagrams 

• Controls: 

- Constraints; GFE, COTS, & Reusable S/W; System concept 
& subsystem choices; organizational procedures 

Controls 

i 


• Activities: 

- Define system states and modes ^ 

- Define system functions & external interfaces Inputs 

- Define functional interfaces 

- Allocate performance requirements to functions 

Functional 
Analysis & 
Allocation 

^^^^-Outputs 

- Analyze performance 

- Analyze timing and resources 

- Analyze failure mode effects and criticality 

- Define fault detection and recovery behavior 

- Integrate functions 

t 

Enablers 



Figure 5-1. Functional Analysis and Allocation 
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activities listed in Figure 5-1, the functional 
architecture becomes more detailed and compre¬ 
hensive. These activities provide a functional 
architecture with sufficient detail to support the 
Design Synthesis. They are performed with the aid 
of traditional tools that structure the effort and pro¬ 
vide documentation for traceability. There are 
many tools available. The following are traditional 
tools that represent and explain the primary tasks 
of Functional Analysis and Allocation (several of 
these are defined and illustrated beginning on page 
49): 

• Functional flow block diagrams that define task 
sequences and relationships, 

• IDEFO diagrams that define process and data 
flows, 

• Timeline analyses that define the time sequence 
of time critical functions, and 

• Requirements allocation sheets that identify 
allocated performance and establish traceability 
of performance requirements. 


5.3 FUNCTIONAL ARCHITECTURE 

The functional architecture is a top-down decom¬ 
position of system functional and performance re¬ 
quirements. The architecture will show not only 
the functions that have to be performed, but also 
the logical sequencing of the functions and 
performance requirements associated with the 
functions. It also includes the functional descrip¬ 
tion of existing and government-furnished items 
to be used in the system. This may require reverse 
engineering of these existing components. 

The functional architecture produced by the 
Functional Analysis and Allocation process is the 
detailed package of documentation developed to 
analyze the functions and allocate performance 
requirements. It includes the functional flow block 
diagrams, timeline sheets, requirements allocation 
sheets, IDEFO diagrams, and all other documenta¬ 
tion developed to describe the functional 
characteristics of the system. However, there is a 
basic logic to the functional architecture, which in 
its preliminary form is presented in the example 
of Figure 5-2. The Functional Analysis and 
Allocation process would normally begin with the 


Perform Mission 


Transport 




50 km 90 min 


First Level: 

Basic Functional 
Requirement 


Second Level: 
Transport and 
communicate 
showing as 
parallel functions 


Third Level: 
Showing decom¬ 
position of the 
transport func¬ 
tion 


A Simple Rule: 

Look to see if all the functions are verbs. If there is a function identified as 
a noun, then there is a problem with the understanding of the functions. 


Communicate 


Required transport r 
requirements allocated 
from mission requirements 


Load 

bH 

Start 


Move 

bH 

Stop 

-► 

Unload 

8 min 


1 min 


75 min 


1 min 

5 min 

0 km 


0 km 


50 km 


0 km 

0 km 


Performance Allocation: 
Performance requirements 
allocated to functions 


Figure 5-2. Functional Architecture Example 
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IPT drafting such a basic version of the archi¬ 
tecture. This would generally give the IPT an 
understanding of the scope and direction of the 
effort. 

Functional Architecture Example 

The Marine Coips has a requirement to transport 
troops in squad-level units over a distance of 50 
kilometers. Troops must be transported within 90 
minutes from the time of arrival of the transport 
system. Constant communication is required dur¬ 
ing the transportation of troops. Figure 5-2 illus¬ 
trates a preliminary functional architecture for this 
simple requirement. 

5.4 SUMMARY POINTS 

Functional analysis begins with the output of 
requirements analysis (that is, the identification of 
higher-level functional and performance require¬ 
ments). Functional Analysis and Allocation con¬ 
sists of decomposition of higher-level functions to 
lower-levels and then allocation of requirements 
to those functions. 


• There are many tools available to support the 
development of a Functional Architecture, such 
as: functional-flow block diagrams, timeline 
analysis sheet, requirements allocation sheet. 
Integrated Definition, and others. 

• Use of the tools illustrated in this chapter is not 
mandatory, but the process they represent is: 

- Define task sequences and relationships 
(functional flow block diagram (FFBD)), 

- Define process and data flows (IDEFO 
diagrams), 

- Define the time sequence of time-critical 
functions (timeline analysis sheets (TLS)), 
and 

- Allocate performance and establish trace- 
ability of performance requirements (require¬ 
ments allocation sheets (RAS)). 
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SUPPLEMENT 5-A 

FUNCTIONAL FLOW 

BLOCK DIAGRAM 

The purpose of the functional flow block diagram 
(FFBD) is to describe system requirements in 
functional terms. 

• Proper sequencing of activities and design 
relationships are established including critical 
design interfaces. 

Objectives 

Characteristics 

The FFBD is structured to ensure that: 

• All life cycle functions are covered. 

• All elements of system are identified and 
defined (e.g. prime equipment, training, spare 
parts, data, software, etc.). 

The FFBD is functionally oriented—not solution 
oriented. The process of defining lower-level func¬ 
tions and sequencing relationships is often referred 
to as functional decomposition. It allows traceabil¬ 
ity vertically through the levels. It is a key step in 
developing the functional architecture from which 
designs may be synthesized. 

• System support requirements are identified to 
specific system functions. 

Figure 5-3 shows the flow-down structure of a set 
of FFBDs and Figure 5-4 shows the format of an 
FFBD. 



Figure 5-3. FFBD Traceability and Indenture 
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Key FFBD Attributes 

Function block: Each function on an FFBD should 
be separate and be represented by single box (solid 
line). Each function needs to stand for definite, 
finite, discrete action to be accomplished by system 
elements. 

Function numbering: Each level should have a 
consistent number scheme and provide informa¬ 
tion concerning function origin. (E.g., top level— 
1.0,2.0,3.0, etc; first indenture (level2)—1.1,1.2, 

1.3, etc; second indenture (level 3)—1.1.1, 1.1.2, 

1.1.3, etc.) These numbers establish identification 
and relationships that will carry through all Func¬ 
tional Analysis and Allocation activities and 
facilitate traceability from lower to top levels. 

Functional reference: Each diagram should con¬ 
tain a reference to other functional diagrams by 
using a functional reference (box in brackets). 


Flow connection: Lines connecting functions 
should only indicate function flow and not a lapse 
in time or intermediate activity. 

Flow direction: Diagrams should be laid out so 
that the flow direction is generally from left to right. 
Arrows are often used to indicate functional flows. 

Summing gates: A circle is used to denote a sum¬ 
ming gate and is used when AND/OR is present. 
AND is used to indicate parallel functions and all 
conditions must be satisfied to proceed. OR is used 
to indicate that alternative paths can be satisfied to 
proceed. 

GO and NO-GO paths: “G” and “bar G” are used 
to denote “go” and “no-go” conditions. These sym¬ 
bols are placed adjacent to lines leaving a particular 
function to indicate alternative paths. 


Abbreviations/Notes: 

“And” Gate: Parallel Function 
“Or” Gate: Alternate Function 


Functional 


function diagrams 
only) 


Scope Note:_ 


Flow level designator- 



►2nd Level 


Title block and standard drawing number- 


Functional Flow Block 
Diagram Format 


Figure 5-4. Functional Flow Block Diagrams (FFBD) Format 
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SUPPLEMENT 5-B 

IDEFO 


Integration Definition for Function Modeling 
(IDEFO) is a common modeling technique for the 
analysis, development, re-engineering, and inte¬ 
gration of information systems; business processes; 
or software engineering analysis. Where the FFBD 
is used to show the functional flow of a product, 
IDEFO is used to show data flow, system control, 
and the functional flow of life cycle processes. 

IDEFO is capable of graphically representing a 
wide variety of business, manufacturing and other 
types of enterprise operations to any level of detail. 
It provides rigorous and precise description, and 
promotes consistency of usage and interpretation. 
It is well-tested and proven through many years of 
use by government and private industry. It can be 
generated by a variety of computer graphics tools. 
Numerous commercial products specifically sup¬ 
port development and analysis of IDEFO diagrams 
and models. 

IDEFO is a model that consists of a hierarchical 
series of diagrams, text, and glossary cross- 


referenced to each other. The two primary model¬ 
ing components are: functions (represented on a 
diagram by boxes), and data and objects that in¬ 
terrelate those functions (represented by arrows). 
As shown by Figure 5-5 the position at which the 
arrow attaches to a box conveys the specific role 
of the interface. The controls enter the top of the 
box. The inputs, the data or objects acted upon by 
the operation, enter the box from the left. The out¬ 
puts of the operation leave the right-hand side of 
the box. Mechanism arrows that provide support¬ 
ing means for performing the function join (point 
up to) the bottom of the box. 

The IDEFO process starts with the identification 
of the prime function to be decomposed. This func¬ 
tion is identified on a “Top Level Context Dia¬ 
gram,” that defines the scope of the particular 
IDEFO analysis. An example of a Top Level Con¬ 
text Diagram for an information system manage¬ 
ment process is shown in Figure 5-6. From this 
diagram lower-level diagrams are generated. An 
example of a derived diagram, called a “child” in 


Input 


Control 


Function Name 


Function 

Number 



Mechanism 


Output 


Figure 5-5. Integration Definition for Function Modeling (IDEFO) Box Format 
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IDEFO terminology, for a life cycle function is ment IDEFO for data intensive systems. The IDEFO 
shown in Figure 5-7. standard, Federal Information Processing Stan¬ 

dards Publication 183 (FIPS 183), and the IDEFlx 
An associated technique. Integration Definition for standard (FIPS 184) are maintained by the National 
Information Modeling (IDEFlx), is used to supple- Institute of Standards and Technology (NIST). 


Program Charter 


Issues 


Operations 
Data 

▲ “ 

Program 

Team 


Purpose: The assessment, planning, and streamlining of information management 
functions. 

Viewpoint: The Information Integration Assessment Team. 


Figure 5-6. Top-Level Context Diagram 


Manage Information Resources 


QA/A-0 
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Maintain Reparable Spares 


Figure 5-7. IDEFO Diagram Example 


pg. 4-5 
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SUPPLEMENT 5-C 

TIMELINE ANALYSIS 
SHEETS 


The timeline analysis sheet (TLS) adds detail to 
defining durations of various functions. It defines 
concurrency, overlapping, and sequential relation¬ 
ships of functions and tasks. It identifies time criti¬ 
cal functions that directly affect system availabil¬ 
ity, operating time, and maintenance downtime. It 
is used to identify specific time-related design 
requirements. 

The TLS includes purpose of function and the 
detailed performance characteristics, criticality of 


function, and design constraints. It identifies 
both quantitative and qualitative performance 
requirements. Initial resource requirements are 
identified. 

Figure 5-8 shows an example of a TLS. The time 
required to perform function 3.1 and its subfunc¬ 
tions are presented on a bar chart showing how the 
timelines relate. (Function numbers match the 
FFBD.) 


Function 3.1 Establish and maintain vehicle 
readiness from 35 hrs to 2 hrs prior to launch. 


Function 

Hours 

Number 

Name 

30 25 20 15 10 

5 A 

3 2 

3.1.1 

Provide ground power 











3.1.2 

Provide vehicle air conditioning 











3.1.3 

Install and connect batteries 



■ 

■ 

■ 






3.1.4 

Install ordnance 

E 


■ 5 








3.1.5 

Perform stray voltage checks and 


■ 

m 









connect ordnance 



y 








3.1.6 

Load fuel tanks 



■ 

" m 







3.1.7 

Load oxidizer tanks 





■ 

■ 





3.1.8 

Activate guidance system 






■ 





3.1.9 

Establish propulsion flight pressure 






i 


|,0 



3.1.10 

Telemetry system “on” 







c 

— 






■ 

■ 

■ 

■ 
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Figure 5-8. Time Analysis Sheet 
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SUPPLEMENT 5-D 

REQUIREMENTS 
ALLOCATION SHEET 


The Requirements Allocation Sheet documents the 
connection between allocated functions, allocated 
performance and the physical system. It provides 
traceability between Functional Analysis and 
Allocation and Design Synthesis, and shows any 


disconnects. It is a major tool in maintaining con¬ 
sistency between functional architectures and de¬ 
signs that are based on them. (Function numbers 
match the FFBD.) 


Requirements 
Allocation Sheet 

Functional Flow Diagram Title and No. 2.58.4 
Provide Guidance Compartment Cooling 

Equipment 

Identification 

Function Name 
and No. 

Functional Performance and 

Design Requirements 

Facility 

Rqmnts 

Nomen¬ 

clature 

Cl or Detail 
Spec No. 

2.58.4 Provide 
Guidance 
Compartment 
Cooling 

The temperature in the guidance 
compartment must be maintained at the 
initial calibration temperature of +0.2 Deg F. 
The initial calibration temperature of the 
compartment will be between 66.5 and 68.5 

Deg F. 




2.58.4.1 Provide 
Chilled Coolant 
(Primary) 

A storage capacity for 65 gal of chilled liquid 
coolant (deionized water) is required. The 
temperature of the stored coolant must be 
monitored continuously. The stored coolant 
must be maintained within a temperature 
range of 40-50 Deg F. for an indefinite 
period of time.The coolant supplied must 
be free of obstructive particles 0.5 micron at 
all times. 





Figure 5-9. Requirements Allocation Sheet (Example) 
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DESIGN SYNTHESIS 


6.1 DESIGN DEVELOPMENT 

Design Synthesis is the process by which concepts 
or designs are developed based on the functional 
descriptions that are the products of Functional 
Analysis and Allocation. Design synthesis is a cre¬ 
ative activity that develops a physical architecture 
(a set of product, system, and/or software elements) 
capable of performing the required functions within 
the limits of the performance parameters pre¬ 
scribed. Since there may be several hardware and/ 
or software architectures developed to satisfy a 
given set of functional and performance require¬ 
ments, synthesis sets the stage for trade studies to 
select the best among the candidate architectures. 
The objective of design synthesis is to combine 


and restructure hardware and software components 
in such a way as to achieve a design solution 
capable of satisfying the stated requirements. 
During concept development, synthesis produces 
system concepts and establishes basic relation¬ 
ships among the subsystems. During preliminary 
and detailed design, subsystem and component 
descriptions are elaborated, and detailed interfaces 
between all system components are defined. 

The physical architecture forms the basis for 
design definition documentation, such as, speci¬ 
fications, baselines, and work breakdown struc¬ 
tures (WBS). Figure 6-1 gives an overview of the 
basic parameters of the synthesis process. 


• Outputs: 

- Physical Architecture (Product Elements and Software Code) 

- Decision Database 


• Inputs: 

- Functional Architecture 


• Enablers: 

- IPTs, Decision Database, Automated Tools, Models 


• Controls: 

- Constraints; GFE, COTS, & Reusable S/W; System concept 
& subsystem choices; organizational procedures 

Controls 

1 

• Activities: 

- Allocate functions and constraints to system elements 

- Synthesize system element alternatives Inputs 

- Assess technology alternatives 

- Define physical interfaces 

- Define system product WBS 

- Develop life cycle techniques and procedures 

- Integrate system elements 

- Select preferred concept/design 

Design 

Synthesis 

^^^^-Outputs 

t 

Enablers 


Figure 6-1. Design Synthesis 
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Characteristics 

Physical architecture is a traditional term. Despite 
the name, it includes software elements as well as 
hardware elements. Among the characteristics of 
the physical architecture (the primary output of 
Design Synthesis) are the following: 

• The correlation with functional analysis 
requires that each physical or software compo¬ 
nent meets at least one (or part of one) func¬ 
tional requirement, though any component can 
meet more than one requirement, 

• The architecture is justified by trade studies and 
effectiveness analyses, 

• A product WBS is developed from the physical 
architecture, 

• Metrics are developed to track progress among 
KPPs, and 

• All supporting information is documented in a 
database. 

Modular Designs 

Modular designs are formed by grouping compo¬ 
nents that perform a single independent function 
or single logical task; have single entry and exit 
points; and arc separately testable. Grouping re¬ 
lated functions facilitates the search for modular 
design solutions and furthermore increases the 
possibility that open-systems approaches can be 
used in the product architecture. 

Desirable attributes of the modular units include 
low coupling, high cohesion, and low connectiv¬ 
ity. Coupling between modules is a measure of their 
interdependence, or the amount of information 
shared between two modules. Decoupling mod¬ 
ules eases development risks and makes later modi¬ 
fications easier to implement. Cohesion (also called 
binding) is the similarity of tasks performed within 
the module. High cohesion is desirable because it 
allows for use of identical or like (family or se¬ 
ries) components, or for use of a single component 
to perform multiple functions. Connectivity refers 


to the relationship of internal elements within one 
module to internal elements within another mod¬ 
ule. High connectivity is undesirable in that it cre¬ 
ates complex interfaces that may impede design, 
development, and testing. 

Design Loop 

The design loop involves revisiting the functional 
architecture to verify that the physical architecture 
developed is consistent with the functional and 
performance requirements. It is a mapping between 
the functional and physical architectures. Figure 
6-2 shows an example of a simple physical archi¬ 
tecture and how it relates to the functional archi¬ 
tecture. During design synthesis, re-evaluation of 
the functional analysis may be caused by the dis¬ 
covery of design issues that require re-examination 
of the initial decomposition, performance alloca¬ 
tion, or even the higher-level requirements. These 
issues might include identification of a promising 
physical solution or open-system opportunities that 
have different functional characteristics than those 
foreseen by the initial functional architecture 
requirements. 

6.2 SYNTHESIS TOOLS 

During synthesis, various analytical, engineering, 
and modeling tools are used to support and 
document the design effort. Analytical devices such 
as trade studies support decisions to optimize 
physical solutions. Requirements Allocation Sheets 
(RAS) provide traceability to the functional and 
performance requirements. Simple descriptions 
like the Concept Decription Sheet (CDS) help visu¬ 
alize and communicate the system concept. Logic 
models, such as the Schematic Block Diagram 
(SBD), establish the design and the interrelation¬ 
ships within the system. 

Automated engineering management tools such as 
Computer-Aided Design (CAD), Computer- 
Aided-Systems Engineering (CASE), and the 
Computer-Aided-Engineering (CAE) can help or¬ 
ganize, coordinate and document the design effort. 
CAD generates detailed documentation describ¬ 
ing the product design including SBDs, detailed 
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-PHYSICAL ARCHITECTURE- 
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Figure 6-2. Functional/Physical Matrix 


drawings, three dimensional and solid drawings, 
and it tracks some technical performance measure¬ 
ments. CAD can provide significant input for vir¬ 
tual modeling and simulations. It also provides a 
common design database for integrated design 
developments. Computer-Aided Engineering can 
provide system requirements and performance 
analysis in support of trade studies, analysis re¬ 
lated to the eight primary functions, and cost analy¬ 
ses. Computer-Aided Systems Engineering can 
provide automation of technical management 
analyses and documentation. 

Modeling 

Modeling techniques allow the physical product 
to be visualized and evaluated prior to design 
decisions. Models allow optimization of hardware 


and software parameters, permit performance 
predictions to be made, allow operational se¬ 
quences to be derived, and permit optimum 
allocation of functional and performance require¬ 
ments among the system elements. The traditional 
logical prototyping used in Design Synthesis is the 
Schematic Block Diagram. 

6.3 SUMMARY POINTS 

• Synthesis begins with the output of Functional 
Analysis and Allocation (the functional archi¬ 
tecture). The functional architecture is trans¬ 
formed into a physical architecture by defining 
physical components needed to perform the 
functions identified in Functional Analysis and 
Allocation. 
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• Many tools are available to support the - Establish traceability of performance 
development of a physical architecture: requirements to components (RAS). 

- Define and depict the system concept (CDS), • Specifications and the product WBS are derived 

from the physical architecture. 

- Define and depict components and their 
relationships (SBD), and 
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SUPPLEMENT 6-A 

CONCEPT DESCRIPTION 
SHEET 


The Concept Description Sheet describes (in tex¬ 
tual or graphical form) the technical approach or 
the design concept, and shows how the system will 


be integrated to meet the performance and func¬ 
tional requirements. It is generally used in early 
concept design to show system concepts. 



External Command Guidance System 


Figure 6-3. Concept Description Sheet Example 
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SUPPLEMENT 6-B 

SCHEMATIC BLOCK 
DIAGRAMS 


The Schematic Block Diagram (SBD) depicts hard¬ 
ware and software components and their interrela¬ 
tionships. They are developed at successively lower 
levels as analysis proceeds to define lower-level 
functions within higher-level requirements. These 
requirements are further subdivided and allocated 
using the Requirements Allocation Sheet (RAS). 
SBDs provide visibility of related system elements, 
and traceability to the RAS, FFBD, and other sys¬ 
tem engineering documentation. They describe a 
solution to the functional and performance require¬ 
ments established by the functional architecture; 
show interfaces between the system components 
and between the system components and other 
systems or subsystems; support traceability 


between components and their functional origin; 
and provide a valuable tool to enhance configura¬ 
tion control. The SBD is also used to develop 
Interface Control Documents (ICDs) and provides 
an overall understanding of system operations. 

A simplified SBD, Figure 6-4, shows how compo¬ 
nents and the connection between them are pre¬ 
sented on the diagram. An expanded version is 
usually developed which displays the detailed func¬ 
tions performed within each component and a de¬ 
tailed depiction of their interrelationships. Ex¬ 
panded SBDs will also identify the WBS numbers 
associated with the components. 



Figure 6-4. Schematic Block Diagram Example 
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SUPPLEMENT 6-C 

REQUIREMENTS ALLOCATION 

SHEET 


The RAS initiated in Functional Analysis and 
Allocation is expanded in Design Synthesis to 
document the connection between functional 
requirements and the physical system. It provides 
traceability between the Functional Analysis and 


Allocation and Synthesis activities. It is a major 
tool in maintaining consistency between functional 
architectures and the designs that are based on 
them. (Configuration Item (Cl) numbers match the 
WBS.) 


Requirements 
Allocation Sheet 

Functional Flow Diagram Title and No. 2.58.4 
Provide Guidance Compartment Cooling 

Equipment 

Identification 

Function Name 
and No. 

Functional Performance and 

Design Requirements 

Facility 

Rqmnts 

Nomenclature 

Cl or Detail 
Spec No. 

2.58.4 Provide 
Guidance 
Compartment 
Cooling 

The temperature in the guidance 
compartment must be maintained 
at the initial calibration tempera¬ 
ture of +0.2 Deg F.The initial cal¬ 
ibration temperature of the com¬ 
partment will be between 66.5 
and 68.5 Deg F. 


Guidance Compart¬ 
ment Cooling 
System 

3.54.5 

2.58.4.1 Provide 
Chilled Coolant 
(Primary) 

A storage capacity for 65 gal of 
chilled liquid coolant (deionized 
water) is required. The temperature 
of the stored coolant must be 
monitored continuously.The stored 
coolant must be maintained within 
a temperature range of 40-50 Deg 

F. for an indefinite period of time. 
The coolant supplied must be free 
of obstructive particles 0.5 micron 
at all times. 


Guidance Compart¬ 
ment Coolant 
Storage Subsystem 

3.54.5.1 


Figure 6-5. Requirements Allocation Sheet (Example) 
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VERIFICATION 


7.1 GENERAL 

The Verification process con fir ms that Design Syn¬ 
thesis has resulted in a physical architecture that 
satisfies the system requirements. Verification rep¬ 
resents the intersection of systems engineering and 
test and evaluation. 

Verification Objectives 

The objectives of the Verification process include 
using established criteria to conduct verification 
of the physical architecture (including software and 
interfaces) from the lowest level up to the total 


system to ensure that cost, schedule, and perfor¬ 
mance requirements are satisfied with acceptable 
levels of risk. Further objectives include generat¬ 
ing data (to confirm that system, subsystem, and 
lower level items meet their specification require¬ 
ments) and validating technologies that will be used 
in system design solutions. A method to verify each 
requirement must be established and recorded dur¬ 
ing requirements analysis and functional alloca¬ 
tion activities. (If it can not be verified it can not 
be a legitimate requirement.) The verification list 
should have a direct relationship to the require¬ 
ments allocation sheet and be continually updated 
to correspond to it. 



Figure 7-1. Systems Engineering and Verification 
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Verification Activities 

System design solutions are verified by the fol¬ 
lowing types of activities: 

1. Analysis - the use of mathematical modeling 
and analytical techniques to predict the com¬ 
pliance of a design to its requirements based 
on calculated data or data derived from lower 
level component or subsystem testing. It is 
generally used when a physical prototype or 
product is not available or not cost effective. 
Analysis includes the use of both modeling and 
simulation which is covered in some detail in 
chapter 13, 

2. Inspection - the visual examination of the sys¬ 
tem, component, or subsystem. It is generally 
used to verify physical design features or 
specific manufacturer identification, 

3. Demonstration - the use of system, subsystem, 
or component operation to show that a require¬ 
ment can be achieved by the system. It is gen¬ 
erally used for a basic confirmation of perfor¬ 
mance capability and is differentiated from test¬ 
ing by the lack of detailed data gathering, or 

4. Test - the use of system, subsystem, or com¬ 
ponent operation to obtain detailed data to 
verify performance or to provide sufficient 
information to verify performance through 
further analysis. Testing is the detailed quan¬ 
tifying method of verification, and as described 
later in this chapter, it is ultimately required in 
order to verify the system design. 

Choice of verification methods must be consid¬ 
ered an area of potential risk. Use of inappropriate 
methods can lead to inaccurate verification. Re¬ 
quired defining characteristics, such as key per¬ 
formance parameters (KPPs) are verified by dem¬ 
onstration and/or test. Where total verification by 
test is not feasible, testing is used to verify key 
characteristics and assumptions used in design 
analysis or simulation. Validated models and simu¬ 
lation tools arc included as analytical verification 
methods that complement other methods. The 
focus and nature of verification activities change 


as designs progress from concept to detailed 
designs to physical products. 

During earlier design stages, verification focuses 
on proof of concept for system, subsystem and 
component levels. During later stages, as the prod¬ 
uct definition effort proceeds, the focus turns to 
verifying that the system meets the customer 
requirements. As shown by Figure 7-1, design is a 
top-down process while the Verification activity is 
a bottom-up process. Components will be fabri¬ 
cated and tested prior to the subsystems. Sub¬ 
systems will be fabricated and tested prior to the 
completed system. 

Performance Verification 

Performance requirements must be objectively 
verifiable, i.e., the requirement must be measur¬ 
able. Where appropriate, Technical Performance 
Measurements (TPM) and other management 
metrics are used to provide insight on progress 
toward meeting performance goals and require¬ 
ments. IEEE Standard PI220 provides a structure 
for Verification activity. As shown in Figure 7-2 
the structure is comprehensive and provides a good 
stalling point for Verification planning. 

7.2 DOD TEST AND EVALUATION 

DoD Test and Evaluation (T&E) policies and pro¬ 
cedures directly support the system engineering 
process of Verification. Testing is the means by 
which objective judgments are made regarding 
the extent to which the system meets, exceeds, 
or fails to meet stated objectives. The puipose of 
evaluation is to review, analyze, and assess data 
obtained from testing and other means to aid in 
making systematic decisions. The puipose of DoD 
T&E is to verify technical performance, opera¬ 
tional effectiveness, operational suitability; and 
it provides essential information in support of 
decision making. 

Common Types of T&E in DoD 

T&E policy requires developmental tests. They 
confirm that technical requirements have been 
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Adapted from IEEE 1220 


Figure 7-2. Verification Tasks 


satisfied, and independent analysis and tests verify 
the system’s operational effectiveness and suita¬ 
bility. DoD T&E traditionally and by directive is 
categorized as: 

• Developmental T&E which focuses primarily 
on technical achievement, 

• Operational T&E which focuses on operational 
effectiveness and suitability and includes Early 
Operational Assessments (EOA), Operational 
Assessment (OA), Initial Operational Test and 


Evaluation (IOT&E), and Follow-On Opera¬ 
tional Test and Evaluation (FOT&E), and 

• Live Fire T&E which provides assessment of 
the vulnerability and lethality of a system by 
subjecting it to real conditions comparable to 
the required mission. 

T&E 

The program office plans and manages the test 

effort to ensure testing is timely, efficient, 
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comprehensive and complete—and that test results 
are converted into system improvements. Test plan¬ 
ning will determine the effectiveness of the 
verification process. Like all systems engineering 
planning activities, careful attention to test 
planning can reduce program risk. The key test 
planning document is the Test and Evaluation 
Master Plan (TEMP). This document lays out the 
objectives, schedule, and resources reflecting pro¬ 
gram office and operational test organization plan¬ 
ning decisions. To ensure integration of this ef¬ 
fort, the program office organizes a Test Planning 
Work Group (TPWG) or Test Working Level IPT 
(WIPT) to coordinate the test planning effort. 

Test Planning Work Group/Test WIPT 

The TPWG/Test WIPT is intended to facilitate the 
integration of test requirements and activities 
through close coordination between the members 
who represent the material developer, designer 
community, logistic community, user, operational 
tester, and other stakeholders in the system devel¬ 
opment. The team outlines test needs based on 
system requirements, directs test design, deter¬ 
mines needed analyses for each test, identifies 
potential users of test results, and provides rapid 
dissemination of test and evaluation results. 

Test and Evaluation Master Plan (TEMP) 

The Test and Evaluation Master Plan is a manda¬ 
tory document prepared by the program office. The 
operational test organization reviews it and 
provides the operational test planning for inclu¬ 
sion. The TEMP is then negotiated between the 
program office and operational test organization. 
After differences are resolved, it is approved at 
appropriate high levels in the stakeholder organi¬ 
zations. After approval it becomes binding on man¬ 
agers and designers (similar to the binding nature 
of the Operational Requirements Document (ORD)). 

The TEMP is a valuable Verification tool that 
provides an excellent template for technology, sys¬ 
tem, and major subsystem-level Verification plan¬ 
ning. The TEMP includes a reaffirmation of the 
user requirements, and to an extent, an interpreta¬ 
tion of what those requirements mean in various 


operational scenarios. Part I of the required TEMP 
format is System Introduction , which provides the 
mission description, threat assessment, MOEs/ 
MOSs, a system description, and an identification 
of critical technical parameters. Part II, Integrated 
Test Program Summary, provides an integrated test 
program schedule and a description of the overall 
test management process. Part III, Developmental 
Test & Evaluation (DT&E) Outline, lays out an 
overview of DT&E efforts and a description of 
future DT&E. Part IV, Operational Test & Evalu¬ 
ation (OT&E) Outline, is provided by the opera¬ 
tional test organization and includes an OT&E 
overview, critical operational issues, future OT&E 
description, and LFT&E description. Part V, Test 
& Evaluation Resource Summary, identifies the 
necessary physical resources and activity respon¬ 
sibilities. This last part includes such items as test 
articles, test sites, test instrumentation, test sup¬ 
port equipment, threat representation, test targets 
and other expendables, operational force test 
support, simulations, models, test-beds, special 
requirements, funding, and training. 

Key Performance Parameters 

Every system will have a set of KPPs that are the 
performance characteristics that must be achieved 
by the design solution. They flow from the opera¬ 
tional requirements and the resulting derived 
MOEs. They can be identified by the user, the 
decision authority, or the operational tester. They 
are documented in the TEMP. 

Developmental Test and Evaluation 

The DT&E verifies that the design solution meets 
the system technical requirements and the system 
is prepared for successful OT&E. DT&E activities 
assess progress toward resolving critical operational 
issues, the validity of cost-performance tradeoff 
decisions, the mitigation of acquisition technical 
risk, and the achievement of system maturity. 

DT&E efforts: 

• Identify potential operational and technologi¬ 
cal capabilities and limitations of the alterna¬ 
tive concepts and design options being pursued; 
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DT&E 



Figure 7-3. DT&E During System Acquisition 


• Support the identification of cost-performance 
tradeoffs by providing analyses of the 
capabilities and limitations of alternatives; 

• Support the identification and description of 
design technical risks; 

• Assess progress toward resolving Critical 
Operational Issues, mitigating acquisition 
technical risk, achieving manufacturing process 
requirements and system maturity; 

• Assess validity of assumptions and analysis 
conclusions; and 

• Provide data and analysis to certify the system 
ready for OT&E, live-fire testing and other 
required certifications. 

Figure 7-3 highlights some of the more signifi¬ 
cant DT&E focus areas and where they fit in the 

acquisition life cycle. 


Live Fire Test and Evaluation 

LFT&E is performed on any Acquisition Category 
(ACAT) I or II level weapon system that includes 
features designed to provide protection to the sys¬ 
tem or its users in combat. It is conducted on a 
production configured article to provide informa¬ 
tion concerning potential user casualties, vulner¬ 
abilities, and lethality. It provides data that can 
establish the system's susceptibility to attack and 
performance under realistic combat conditions. 

Operational Test and Evaluation 

OT&E programs are structured to determine the 
operational effectiveness and suitability of a sys¬ 
tem under realistic conditions, and to determine if 
the minimum acceptable operational performance 
requirements as specified in the ORD and reflected 
by the KPPs have been satisfied. OT&E uses threat- 
representative forces whenever possible, and em¬ 
ploys typical users to operate and maintain the 
system or item under conditions simulating both 
combat stress and peacetime conditions. Opera¬ 
tional tests will use production or production- 


69 







Systems Engineering Fundamentals 


Chapter 7 



EOA 


Figure 7-4. OT&E During System Acquisition 


representative articles for the operational tests that 
support the full-rate production decision. Live Fire 
Tests are usually performed during the operational 
testing period. Figure 7-4 shows the major activi¬ 
ties associated with operational testing and where 
they fit in the DoD acquisition life cycle. 

OT&E Differences 

Though the overall objective of both DT&E and 
OT&E is to verify the effectiveness and suitability 
of the system, there are distinct differences in their 
specific objects and focus. DT&E primarily fo¬ 
cuses on verifying system technical requirements, 
while OT&E focuses on verifying operational re¬ 
quirements. DT&E is a program office responsi¬ 
bility that is used to develop the design. OT&E is 
an independent evaluation of design maturity that 


is used to determine if the program should pro¬ 
ceed to full-rate production. Figure 7-5 lists the 
major differences between the two. 

7.3 SUMMARY POINTS 

The Verification activities of the Systems Engineer¬ 
ing Process are performed to verify that physical 
design meets the system requirements. 

• DoD T&E policy supports the verification pro¬ 
cess through a sequence of Developmental, 
Operational, and Live-Fire tests, analyses, and 
assessments. The primary management tools for 
planning and implementing the T&E effort are 
the TEMP and the integrated planning team. 
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Development Tests 

Operational Tests 

• Controlled by program manager 

• Controlled by independent agency 

• One-on-one tests 

• Many-on-many tests 

• Controlled environment 

• Realistic/tactical environment with 

• Contractor environment 

• Trained, experienced operators 

• Precise performance objectives and 
threshold measurements 

operational scenario 

• No system contractor involvement 

• User troops recently trained 

• Performance measures of operational 

• Test to specification 

• Developmental, engineering, or production 
representative test article 

effectiveness and suitability 

• Test to operational requirements 

• Production representative test article 


Figure 7-5. DT/OT Comparison 
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SYSTEMS ENGINEERING 
PROCESS OUTPUTS 


8.1 DOCUMENTING REQUIREMENTS 
AND DESIGNS 

Outputs of the systems engineering process con¬ 
sist of the documents that define the system re¬ 
quirements and design solution. The physical 
architecture developed through the synthesis pro¬ 
cess is expanded to include enabling products and 
services to complete the system architecture. This 
system level architecture then becomes the refer¬ 
ence model for further development of system re¬ 
quirements and documents. System engineering 
process outputs include the system and configura¬ 
tion item architectures, specifications, and 
baselines, and the decision database. 

Outputs are dependent on the level of development. 
They become increasingly technically detailed as 
system definition proceeds from concept to detailed 
design. As each stage of system definition is 
achieved, the information developed forms the 
input for succeeding applications of the system 
engineering process. 

Architectures: System/Configuration Item 

The System Architecture describes the entire sys¬ 
tem. It includes the physical architecture produced 
through design synthesis and adds the enabling 
products and services required for life cycle 
employment, support, and management. Military 
Handbook (MIL-HDBK)-881, Work Breakdown 
Structures , provides reference models for weapon 
systems architectures. As shown by Figure 8-1, 
MIL-HDBK-881 illustrates the first three levels 
of typical system architectures. Program Offices 
can use MIL-HDBK-881 templates during system 
definition to help develop a top-level architec¬ 
ture tailored to the needs of the specific system 


considered. The design contractor will normally 
develop the levels below these first three. Chapter 9 
of this text describes the WBS in more detail. 

Specifications 

A specification is a document that clearly and 
accurately describes the essential technical require¬ 
ments for items, materials, or services including 
the procedures by which it can be determined that 
the requirements have been met. Specifications 
help avoid duplication and inconsistencies, allow 
for accurate estimates of necessary work and 
resources, act as a negotiation and reference docu¬ 
ment for engineering changes, provide documen¬ 
tation of configuration, and allow for consistent 
communication among those responsible for the 
eight primary functions of Systems Engineering. 
They provide IPTs a precise idea of the problem 
to be solved so that they can efficiently design the 
system and estimate the cost of design alternatives. 
They provide guidance to testers for verification 
(qualification) of each technical requirement. 

Program-Unique Specifications 

During system development a series of specifica¬ 
tions are generated to describe the system at dif¬ 
ferent levels of detail. These program unique speci¬ 
fications form the core of the configuration 
baselines. As shown by Figure 8-2, in addition to 
referring to different levels within the system hi¬ 
erarchy, these baselines are defined at different 
phases of the design process. 

Initially the system is described in terms of the 
top-level (system) functions, performance, and in¬ 
terfaces. These technical requirements are derived 
from the operational requirements established by 
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Level 1 


Aircraft System 


Level 2 


Air 

Vehicle 

SE/ 

Program 

Mgmt 

System 

T&E 

Training 

Data 

Peculiar 

Support 

Equipment 

Common 

Support 

Equipment 

Op/Site 

Activation 

Industrial 

Facilities 

Initial 

Spares and 
Initial 
Repair 
Parts 

Airframe DT&E Equipment Tech Pubs Test and Test and Sys Construe- 


Propulsion 
Application Software 
System Software 
C3I 

Navigation/Guidance 
Central Computer 
Fire Control 

Data Display and Controls 
Survivability 
Reconnaissance 
Automatic Flight Control 
Central Integrated Checkout 
Antisubmarine Warfare 
Armament 


OT&E 

Mockups 

T&E 

Support 

Test 

Facilities 


Services 

Facilities 


Engrg Data 

Support 

Data 

Manage¬ 
ment Data 

Data 

Depository 


Measurem’t 

Equipment 

Support 

and 

Handling 

Equipment 


Measurem’t 

Equipment 

Support 

and 

Handling 

Equipment 


Assembly, 

Installation 

and 

Checkout 
on Site 

Contractor 
Tech Support 

Site 

Construction 

Site/Ship 

Vehicle 

Conversion 


tion/Conver- 

sion/Expan- 

sion 

Equipment 
Acquisition 
or Mod 

Maintenance 


(Specify by 
Allowance 
List, 

Grouping 
or H/W 
Element) 


Level 3 


Weapons Delivery 
Auxiliary Equipment 


Figure 8-1. Example from MIL-HDBK-881 


the user. This system-level technical description is 
documented in the System Specification, which is 
the primary documentation of the system-level 
Functional Baseline. The system requirements are 
then flowed down (allocated) to the items below 
the system level, such that a set of design criteria 
are established for each of those items. These item 
descriptions are captured in a set of Item Perfor¬ 
mance Specifications, which together with other 
interface definitions, process descriptions, and 
drawings, document the Allocated Baseline (some¬ 
times referred to as the “Design To” baseline). 
Having baselined the design requirements for the 
individual items, detailed design follows. Detailed 
design involves defining the system from top to 
bottom in terms of the physical entities that will 
be employed to satisfy the design requirements. 
When detailed design is complete, a final baseline 
is defined. This is generally referred to as the Prod¬ 
uct Baseline, and, depending on the stage of de¬ 
velopment, may reflect a “Build to” or “As built” 
description. The Product Baseline is documented 


by the Technical Data Package, which will include 
not only Item Detail Specifications, but also. Pro¬ 
cess and Material Specifications, as well as draw¬ 
ings, parts lists, and other information that de¬ 
scribes the final system in full physical detail. Fig¬ 
ure 8-3 shows how these specifications relate to 
their associated baselines. 

Role of Specifications 

Requirements documents express why the devel¬ 
opment is needed. Specification documents are an 
intermediate expression of what the needed sys¬ 
tem has to do in terms of technical requirements 
(function, performance, and interface). Design 
documents (drawings, associated lists, etc.) de¬ 
scribe the means by which the design requirements 
are to be satisfied. Figure 8-4 illustrates how 
requirements flow down from top-level specifica¬ 
tions to design documentation. Preparation of 
specifications are paid of the system engineering 
process, but also involve techniques that relate to 
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System 
Concept p 


System Definition (Functional Baseline) 


Integrated and 
Qualified System 



Figure 8-2. Specifications and Levels of Development 


Specification 

Content 

Baseline 

System 

Spec 

Defines mission/technical performance requirements. 

Allocates requirements to functional areas and defines interfaces. 

Functional 

Item 

Performance 

Spec 

Defines performance characteristics of CIs and CSCIs. 

Details design requirements and with drawings and other 
documents form the Allocated Baseline. 

Allocated 
“Design To” 

Item Detail 
Spec 

Defines form, fit, function, performance, and test requirements 
for acceptance. (Item, process, and material specs start the 

Product Baseline effort, but the final audited baseline includes 
all the items in the TDP.) 

Product 
“Build To” 

or 

“As Built” 

Process 

Spec 

Defines process performed during fabrication. 


Material 

Spec 

Defines production of raw materials or semi-fabricated 
material used in fabrication. 



Figure 8-3. Specification Types 
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communication skills, both legal and editorial. 
Figure 8-5 provides some rules of thumb that 
illustrate this. 

In summary, specifications document what the 
system has to do, how well it has to do it, and how 
to verify it can do it. 

Baselines 

Baselines formally document a product at some 
given level of design definition. They are refer¬ 
ences for the subsequent development to follow. 
Most DoD systems are developed using the three 
classic baselines described above: functional, 
allocated, and product. Though the program unique 
specifications arc the dominant baseline documen¬ 
tation, they alone do not constitute a baseline. 

Additional documents include both end and en¬ 
abling product descriptions. End product baseline 
documents normally include those describing 
system requirements, functional architecture, 
physical architecture, technical drawing package, 


and requirements traceability. Enabling product 
baseline documents include a wide range of 
documents that could include manufacturing plans 
and processes, supportability planning, supply 
documentation, manuals, training plans and pro¬ 
grams. test planning, deployment planning, and 
others. All enabling products should be reviewed 
for their susceptibility to impact from system con¬ 
figuration changes. If a document is one that 
describes a part of a system and could require 
change if the configuration changes, then most 
likely it should be included as a baseline document. 

Acquisition Program Baselines 

Acquisition Program Baselines and Configuration 
Baselines are related. To be accurate the Program 
baseline must reflect the realities of the Configu¬ 
ration Baseline, but the two should not be con¬ 
fused. Acquisition Program Baselines are high level 
assessments of program maturity and viability. 
Configuration Baselines are system descriptions. 
Figure 8-6 provides additional clarification. 



Figure 8-4. How Specifications Lead to Design Documents 
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• Use a table of contents and define all abbreviations and acronyms. 

• Use active voice. 

• Use “shall” to denote mandatory requirement and “may” or “should” to denote guidance 
provisions. 

• Avoid ambiguous provisions, such as “as necessary,” “contractor’s best practice,” “smooth 
finish,” and similar terms. 

• Use the System Engineering Process to identify requirements. Do not over-specify. 

• Avoid “tiering.” Any mandatory requirement in a document below the first tier, should be stated 
in the specification. 

• Only requirement sections of the MIL-STD-491D formats are binding. Do not put requirements 
in non-binding sections, such as Scope, Documents, or Notes. 

• Data documentation requirements are specified in a Contract Data Requirements List. 

Figure 8-5. Rules of Thumb for Specification Preparation 


Decision Database 

The decision database is the documentation that 
supports and explains the configuration solution 
decisions. It includes trade studies, cost effective¬ 
ness analyses. Quality Function Deployment (QFD) 
analysis, models, simulations, and other data 
generated to understand a requirement, develop 
alternative solutions, or make a choice between 
them. These items are retained and controlled as 
part of the Data Management process described in 
Chapter 10. 


8.2 DOD POLICY AND PRACTICE- 
SPECIFICATIONS AND STANDARDS 

DoD uses specifications to communicate product 
requirements and standards to provide guidance 
concerning proven methods and practices. 

Specifications 

DoD uses three basic classifications of specifica¬ 
tions: materiel specifications (developed by DoD 
components), Program-Unique Specifications, and 
non-DoD specifications. 


• Program Baselines 

- Embody only the most important cost, 
schedule, and performance objectives 
and thresholds 

- Threshold breach results in re-evalua- 
tion of program at MDA level 

- Selected key performance parameters 

- Specifically evolves over the develop¬ 
ment cycle and is updated at each major 
milestone review or program restructure 

• Required on ALL programs for measuring 
and reporting status 


• Configuration Baselines 

Identify and define an item’s functional 
and physical characteristics 

- Functional Baseline - Describes system 
level requirements 

- Allocated Baseline - Describes design 
requirements for items below system 
level 

- Product Baseline- Describes product 
physical detail 

• Documents outputs of Systems Engineering 
Process 


Figure 8-6. Acquisition Program Baselines and Configuration Baselines 
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DoD developed specifications describe essential 
technical requirements for purchase of materiel. 
Program-Unique Specifications are an integral paid 
of the system development process. Standard prac¬ 
tice for preparation of DoD and Program-Unique 
Specifications is guided by MIL-STD-961D. 
This standard provides guidance for the develop¬ 
ment of performance and detail specifications. 
MIL-STD-961D, Appendix A provides further 
guidance for the development of Program-Unique 
Specifications. 

Non-DoD specifications and standards approved 
for DoD use are listed in the DoD Index of 
Specifications and Standards (DoDISS). 

DoD Policy (Specifications) 

DoD policy is to develop performance specifica¬ 
tions for procurement and acquisition. In general, 
detail specifications are left for contractor devel¬ 
opment and use. Use of a detail specification in 
DoD procurement or acquisition should be con¬ 
sidered only where absolutely necessary, and then 
only with supporting trade studies and acquisition 
authority approval. 

DoD policy gives preference to the use of com¬ 
mercial solutions to government requirements, 
rather than development of unique designs. There¬ 
fore, the use of commercial item specifications and 
descriptions should be a priority in system archi¬ 
tecture development. Only when no commercial 
solution is available should government detail 
specifications be employed. 

In the case of re-procurement, where detail speci¬ 
fications and drawings are government owned, 
standardization or interface requirements may 
present a need for use of detailed specifications. 
Trade studies that reflect total ownership costs and 
the concerns related to all eight primary functions 
should govern decisions concerning the type of 
specification used for re-procurement of systems, 
subsystems, and configuration items. Such trade 
studies and cost analysis should be prefoimed prior 
to the use of detail specifications or the decision 


to develop and use performance specifications in 
a reprocurement. 

Performance Specifications 

Performance Specifications state requirements in 
terms of the required results with criteria for veri¬ 
fying compliance, but without stating the methods 
for achieving the required results. In general, per¬ 
formance specifications define products in terms 
of functions, performance, and interface require¬ 
ments. They define the functional requirements for 
the item, the environment in which it must oper¬ 
ate, and interface and interchangeability charac¬ 
teristics. The contractor is provided the flexibility 
to decide how the requirements are best achieved, 
subject to the constraints imposed by the govern¬ 
ment, typically through interface requirements. 
System Specifications and Item Performance 
Specifications are examples of performance 
specifications. 

Detail Specifications 

Detail Specifications, such as Item Detail, Mate¬ 
rial and Process Specifications, provide design re¬ 
quirements. This can include materials to be used, 
how a requirement is to be achieved, or how an 
item is to be fabricated or constructed. If a specifi¬ 
cation contains both performance and detail re¬ 
quirements, it is considered a Detail Specification, 
with the following exception: Interface and inter¬ 
changeability requirements in Performance Speci¬ 
fications may be expressed in detailed terms. For 
example, a Performance Specification for shoes 
would specify size requirements in detailed terms, 
but material or method of construction would be 
stated in performance terms. 

Software Documentation - IEEE/EIA 12207 

IEEE/EIA 12207, Software Life Cycle Processes, 
describes the U.S. implementation of the ISO stan¬ 
dard on software processes. This standard describes 
the development of software specifications as one 
aspect of the software development process. 
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The process described in IEEE/EIA 12207 for 
allocating requirements in a top-down fashion and 
documenting the requirements at all levels parallels 
the systems engineering process described in this 
text. The standard requires first that system-level 
requirements be allocated to software items (or 
configuration items) and that the software 
requirements then be documented in terms of func¬ 
tionality. performance, and interfaces, and that 
qualification requirements be specified. Software 
item requirements must be traceable to system- 
level, and be consistent and verifiable. 

The developer is then required to decompose each 
software item into software components and then 
into software units that can be coded. Requirements 
are allocated from item level, to component, and 
finally to unit level. This is the detailed design 
activity and IEEE/EIA 12207 requires that these 
allocations of requirements be documented in 
documents that are referred to as “descriptions,” 
or, if the item is a “stand alone” item, as “specifi¬ 
cations.” The content of these documents is defined 
in the IEEE/EIA standard; however, the level of 
detail required will vary by project. Each project 
must therefore ensure that a common level of 


expectation is established among all stakeholders 
in the software development activity. 

Standard Practice for Defense Specifications - 
MIL-STD-961D 

The puipose of MIL-STD-961D is to establish 
uniform practices for specification preparation, to 
ensure inclusion of essential requirements, to 
ensure Verification (qualification) methods are es¬ 
tablished for each requirement, and to aid in the 
use and analysis of specification content. MIL- 
STD-961D establishes the format and content of 
system, configuration item, software, process and 
material specifications. These Program-Unique 
Specifications are developed through application 
of the systems engineering process and represent 
a hierarchy as shown in Figure 8-7. 

Standards 

Standards establish engineering and technical 
limitations and applications for items, materials, 
processes, methods, designs, and engineering 
practices. They are “corporate knowledge” docu¬ 
ments describing how to do some process or a 



Software Product Spec 

• Software Design Description 

• Interface Design Description 


Figure 8-7. Specification Hierarchy 


79 










Systems Engineering Fundamentals 


Chapter 8 


description of a body of knowledge. Standards 
come from many sources, reflecting the practices 
or knowledge base of the source. Format and con¬ 
tent of Defense Standards, including Flandbooks, 
are governed by MIL-STD-962. Other types of 
standards in use in DoD include Commercial Stan¬ 
dards, Coiporate Standards, International Stan¬ 
dards, Federal Standards, and Federal Information 
Processing Standards. 

DoD Policy (Standards) 

DoD policy does not require standard management 
approaches or manufacturing processes on con¬ 
tracts. This policy applies to the imposition of both 
Military Specifications and Standards and, in ad¬ 
dition, to the imposition of Commercial and In¬ 
dustry Standards. In general, the preferred ap¬ 
proach is to allow contractors to use industry, gov¬ 
ernment, corporate, or company standards they 
have determined to be appropriate to meet 
government’s needs. The government reviews and 
accepts the contractor’s approach through a 
contract selection process or a contractual review 
process. 

The government should impose a process or 
standard only as a last resort, and only with the 
support of an appropriate trade study analysis. If a 
specific standard is imposed in a solicitation or 
contract, a waiver will be required from an 
appropriate Service authority. 

However, there is need on occasion to direct the 
use of some standards for reasons of standardiza¬ 
tion, interfaces, and development of open systems. 
A case in point is the mandated use of the Joint 
Technical Architecture (JTA) for defining 
interoperability standards. The JTA sets forth the 
set of interface standards that are expected to be 
employed in DoD systems. The JTA is justifiably 
mandatory because it promotes needed 
interoperability standardization, establishes sup¬ 
portable interface standards, and promotes the 
development of open systems. 

DoD technical managers should be alert to situa¬ 
tions when directed standards are appropriate to 
their program. Decisions concerning use of 


directed standards should be confirmed by trade 
studies and requirements traceability. 

DoD Index of Specifications and Standards 

The DoDISS lists all international, adopted indus¬ 
try standardization documents authorized for use 
by the military departments, federal and military 
specifications and standards. Published in three 
volumes, it contains over 30,000 documents in 103 
Federal Supply Groups broken down into 850 Fed¬ 
eral Supply Classes. It covers the total DoD use of 
specifications and standards, ranging from fuel 
specifications to international quality standards. 

8.3 SUMMARY POINTS 

• System Engineering Process Outputs include 
the system/configuration item architecture, 
specifications and baselines, and the decision 
database. 

• System/Configuration Item Architectures in¬ 
clude the physical architecture and the associ¬ 
ated products and services. 

• Program-Unique specifications are a primary 
output of the System Engineering Process. Pro¬ 
gram-Unique specifications describe what the 
system or configuration item must accomplish 
and how it will be verified. Program-Unique 
specifications include the System, Item Perfor¬ 
mance, and Item Detail Specifications. The 
System Specification describes the system re¬ 
quirements, while Item Performance and Item 
Detail Specifications describe configuration 
item requirements. 

• Configuration baselines are used to manage and 
control the technical development. Program 
baselines arc used for measuring and supporting 
program status. 

• The Decision Database includes those docu¬ 
ments or software that support understanding 
and decision making during formulation of the 
configuration baselines. 
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DoD policy is to develop performance specifi¬ 
cations for procurement and acquisition. Use 
of other than performance specifications in a 
contract must be justified and approved. 

It is DoD policy not to require standard manage¬ 
ment approaches or manufacturing processes 
on contracts. 


Mandatory use of some standard practices are 
necessary, but must be justified through 
analysis. A case in point is the mandatory use 
of the standards listed in the Joint Technical 
Architecture. 
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CHAPTER 9 


WORK BREAKDOWN 
STRUCTURE 


9.1 INTRODUCTION 

The Work Breakdown Structure (WBS) is a means 
of organizing system development activities based 
on system and product decompositions. The sys¬ 
tems engineering process described in earlier chap¬ 
ters produces system and product descriptions. 
These product architectures, together with associ¬ 
ated services (e.g., program management, systems 
engineering, etc.) are organized and depicted in a 
hierarchical tree-like structure that is the WBS. 
(See Figure 9-1.) 

Because the WBS is a direct derivative of the physi¬ 
cal and systems architectures it could be consid¬ 
ered an output of the systems engineering process. 
It is being presented here as a Systems Analysis 
and Control tool because of its essential utility for 
all aspects of the systems engineering process. It 


is used to structure development activities, to iden¬ 
tify data and documents, and to organize integrated 
teams, and for other non-technical program 
management puiposes. 

WBS Role in DoD Systems Engineering 

DoD 5000.2-R requires that a program WBS be 
established to provide a framework for program 
and technical planning, cost estimating, resource 
allocation, performance measurement, and status 
reporting. The WBS is used to define the total 
system, to display it as a product-oriented family 
tree composed of hardware, software, services, 
data, and facilities, and to relate these elements to 
each other and to the end product. Program offices 
are to tailor a program WBS using the guidance 
provided in MIL-HDBK-881. 


Architecture 


WBS 


WBS Elements 







1000 Aircraft Subsystems 
1610 Landing Gear 


Figure 9-1. Architecture to WBS Flow 
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The program WBS is developed initially to define 
the top three levels. As the program proceeds 
through development and is further defined, pro¬ 
gram managers should ensure that the WBS is 
extended to identify all high-cost and high-risk 
elements for management and reporting, while 
ensuring the contractor has complete flexibility to 
extend the WBS below the reporting requirement 
to reflect how work will be accomplished. 

Basic Purposes of the WBS 

Organizational: 

The WBS provides a coordinated, complete, and 
comprehensive view of program management. It 
establishes a structure for organizing system 
development activities, including IPT design, 
development, and maintenance. 

Business: 

It provides a structure for budgets and cost esti¬ 
mates. It is used to organize collection and analy¬ 
sis of detailed costs for earned value reports (Cost 
Performance Reports or Cost/Schedule Control 
System Criteria reporting). 

Technical: 

The WBS establishes a structure for: 

• Identifying products, processes, and data, 

• Organizing risk management analysis and 
tracking, 

• Enabling configuration and data management. 
It helps establish interface identification and 
control. 

• Developing work packages for work orders and 
material/part ordering, and 

• Organizing technical reviews and audits. 

The WBS is used to group product items for speci¬ 
fication development, to develop Statements of 
Work (SOW), and to identify specific contract 
deliverables. 


WBS - Benefits 

The WBS allows the total system to be described 
through a logical breakout of product elements into 
work packages. A WBS, correctly prepared, will 
account for all program activity. It links program 
objectives and activities with resources, facilitates 
initial budgets, and simplifies subsequent cost 
reporting. The WBS allows comparison of vari¬ 
ous independent metrics and other data to look for 
comprehensive trends. 

It is a foundation for all program activities, includ¬ 
ing program and technical planning, event sched¬ 
ule definition, configuration management, risk 
management, data management, specification 
preparation, SOW preparation, status reporting 
and problem analysis, cost estimates, and budget 
formulation. 


9.2 WBS DEVELOPMENT 

The physical and system architectures are used to 
prepare the WBS. The architectures should be 
reviewed to ensure that all necessary products and 
services are identified, and that the top-down struc¬ 
ture provides a continuity of flow down for all 
tasks. Enough levels must be provided to identify 
work packages for cost/schedule control purposes. 
If too few levels are identified, then management 
visibility and integration of work packages may 
suffer. If too many levels are identified, then pro¬ 
gram review and control actions may become 
excessively time-consuming. 

The first three WBS Levels are organized as: 
Level 1 - Overall System 
Level 2 - Major Element (Segment) 

Level 3 - Subordinate Components (Prime 
Items) 

Levels below the first three represent component 
decomposition down to the configuration item 
level. In general, the government is responsible for 
the development of the first three levels, and the 
contractor(s) for levels below three. 
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DoD Practice 

In accordance with DoD mandatory procedures in 
DoD 5000.2-R and common DoD practice as es¬ 
tablished in MIL-HDBK-881, the program office 
develops a program WBS and a contract WBS for 
each contract. The program WBS is the WBS that 
represents the total system, i.e., the WBS that 
describes the system architecture. The contract 
WBS is the part of the program WBS that relates 
to deliverables and tasks of a specific contract. 

MIL-HDBK-881 is used by the program office to 
support the systems engineering process in devel¬ 
oping the first three levels of the program WBS, 
and to provide contractors with guidance for lower 
level WBS development. As with most standards 
and handbooks, use of MIL-HDBK-881 cannot be 
specified as a contract requirement. 

Though WBS development is a systems engineer¬ 
ing activity, it impacts cost and budget profession¬ 
als, as well as contracting officers. An integrated 
team representing these stakeholders should be 
formed to support WBS development. 

WBS Anatomy 

A program WBS has an end product paid and an 
enabling product part. The end product part of the 


system typically consists of the prime mission 
product(s) delivered to the operational customer. 
This paid of the WBS is based on the physical 
architectures developed from operational require¬ 
ments. It represents that part of the WBS involved 
in product development. Figure 9-2 presents a 
simple example of a program WBS product paid. 

The “enabling product” paid of the system includes 
the products and services required to develop, 
produce, and support the end product(s). This part 
of the WBS includes the horizontal elements of 
the system architecture (exclusive of the end prod¬ 
ucts), and identifies all the products and services 
necessary to support the life cycle needs of the 
product. Figure 9-3 shows an example of the top 
three levels of a complete WBS tree. 

Contract WBS 

A contract WBS is developed by the program office 
in preparation for contracting for work required to 
develop the system. It is further developed by the 
contractor after contract award. The contract WBS 
is that portion of the program WBS that is specifi¬ 
cally being tasked through the contract. A simple 
example of a contract WBS derived from the 
program WBS shown in Figure 9-2 is provided by 
Figure 9-4. Figure 9-4, like Figure 9-2, only 
includes the product paid of the contract WBS. A 



Figure 9-2. Program WBS -The Product Part (Physical Architecture) 
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Figure 9-3. The Complete Work Breakdown Structure 


complete contract WBS would include associated 
enabling products, similar to those identified in 
Figure 9-3. The resulting complete contract WBS 


is used to organize and identify contractor tasks. 
The program office’s preliminary version is used 
to develop a SOW for the Request for Proposals. 



Figure 9-4. Contract WBS 
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9.3 DESIGNING AND TRACKING WORK 

A prime use of the WBS is the design and tracking 
of work. The WBS is used to establish what work 
is necessary, a logical decomposition down to work 
packages, and a method for organizing feedback. 
As shown by Figure 9-5, the WBS element is 
matrixed against those organizations in the com¬ 
pany responsible for the task. This creates cost 
accounts and task definition at a detailed level. It 
allows rational organization of integrated teams 
and other organizational structures by helping 
establish what expertise and functional support is 
required for a specific WBS element. It further 
allows precise tracking of technical and other 
management. 


WBS Dictionary 

As part of the work and cost control use of the 
WBS, a Work Breakdown Dictionary is developed. 
For each WBS element a dictionary entry is pre¬ 
pared that describes the task, what costs (activi¬ 
ties) apply, and the references to the associated 
Contract Line Item Numbers and SOW paragraph. 
An example of a level 2 WBS element dictionary 
entry is shown as Figure 9-6. 

9.4 SUMMARY POINTS 

• The WBS is an essential tool for the organiza¬ 
tion and coordination of systems engineering 
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Index Item No. 2 WBS Level 2 

CONTRACT NUMBER 

F33657-72-C-0923 

WBS Element 

A10100 

WBS Title 

Air Vehicle 

Contract 

Line Item: 

0001,0001AA, 0001AB, 0001 AC, 0001 AD 

0001AE, 0001AF, 0001 AG, 0001 AH 

Date 

Chg 

Revision No. 

Revision Auth 

Approved 

Specification No. 

689E078780028 

Specification Title: 

Prime Item Development 

Specification for AGM 86A Air Vehicle/ 
Airframe 

Element Task Description 

Technical Content: 

The Air Vehicle element task description refers to the effort 
required to develop, fabricate, integrate and test the 
airframe segment, portions of the Navigation/Guidance 
element, and Airborne Development Test Equipment and 
Airborne Operational Test Equipment and to the integra¬ 
tion assembly and check-out of these complete elements, 
together with the Engine Segment, to produce the 
complete Air Vehicle. The lower-level elements included 
and summarized in the Air Vehicle element are: 

Airframe Segment (A11100), Navigation/Guidance 
Segment (A32100), Airborne Development Test 

Equipment (A61100), and Airborne Operational Test 
Equipment (A61200). 

Cost Description 

MPC/PMC Work Order/Work Auth 

A10100 See lower level 

WBS Elements 

Cost Content - System Contractor 

The cost to be accumulated against this element includes 
a summarization of all costs required to plan, develop, 
fabricate, assemble, integrate and perform development 
testing, analysis and reporting for the air vehicle. It also 
includes all costs associated with the required efforts in 
integrating, assembling and checking our GFP required to 
create this element. 

Applicable SOW Paragraph 

3.6.2 


Figure 9-6. Work Breakdown Dictionary 


processes, and it is a product of the systems 
engineering process. 

• Its importance extends beyond the technical 
community to business professionals and con¬ 
tracting officers. The needs of all stakeholders 
must be considered in its development. The pro¬ 
gram office develops the program WBS and a 
high-level contract WBS for each contract. The 


contractors develop the lower levels of the 
contract WBS associated with their contract. 

• The system architecture provides the structure 
for a program WBS. SOW tasks flow from this 
WBS. 

• The WBS provides a structure for organizing 
IPTs and tracking metrics. 
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CONFIGURATION 

MANAGEMENT 


10.1 FOUNDATIONS 
Configuration Defined 

A “configuration” consists of the functional, physi¬ 
cal, and interface characteristics of existing or 
planned hardware, firmware, software or a combi¬ 
nation thereof as set forth in technical documenta¬ 
tion and ultimately achieved in a product. The con¬ 
figuration is formally expressed in relation to a 
Functional, Allocated, or Product configuration 
baseline as described in Chapter 8. 

Configuration Management 

Configuration management permits the orderly 
development of a system, subsystem, or configu¬ 
ration item. A good configuration management pro¬ 
gram ensures that designs are traceable to require¬ 
ments, that change is controlled and documented, 
that interfaces are defined and understood, and that 
there is consistency between the product and its 
supporting documentation. Configuration manage¬ 
ment provides documentation that describes what 
is supposed to be produced, what is being produced, 
what has been produced, and what modifications 
have been made to what was produced. 

Configuration management is performed on 
baselines, and the approval level for configuration 
modification can change with each baseline. In a 
typical system development, customers or user 
representatives control the operational require¬ 
ments and usually the system concept. The devel¬ 
oping agency program office normally controls the 
functional baseline. Allocated and product base¬ 
lines can be controlled by the program office, the 
producer, or a logistics agent depending on the life 
cycle management strategy. This sets up a hierarchy 


of configuration control authority corresponding 
to the baseline structure. Since lower level baselines 
have to conform to a higher-level baseline, changes 
at the lower levels must be examined to assure they 
do not impact a higher-level baseline. If they do, 
they must be approved at the highest level im¬ 
pacted. For example, suppose the only engine 
turbine assembly affordably available for an engine 
development cannot provide the continuous oper¬ 
ating temperature required by the allocated base¬ 
line. Then not only must the impact of the change 
at the lower level (turbine) be examined, but the 
change should also be reviewed for possible im¬ 
pact on the functional baseline, where requirements 
such as engine power and thrust might reside. 

Configuration management is supported and 
performed by integrated teams in an Integrated 
Product and Process Development (IPPD) envi¬ 
ronment. Configuration management is closely 
associated with technical data management and 
interface management. Data and interface manage¬ 
ment is essential for proper configuration manage¬ 
ment, and the configuration management effort has 
to include them. 

DoD Application of 
Configuration Management 

During the development contract, the Government 
should maintain configuration control of the 
functional and performance requirements only, 
giving contractors responsibility for the detailed 
design. (SECDEF Memo of 29 Jun 94.) This im¬ 
plies government control of the Functional (sys¬ 
tem requirements) Baseline. Decisions regarding 
whether or not the government will take control of 
the lower-level baselines (allocated and product 
baselines), and when ultimately depends on the 
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requirements and strategies needed for the particu¬ 
lar program. In general, government control of 
lower-level baselines, if exercised, will take place 
late in the development program after design has 
stabilized. 

Configuration Management Planning 

When planning a configuration management ef¬ 
fort you should consider the basics: what has to be 
done, how should it be done, who should do it, 
when should it be done, and what resources are 
required. Planning should include the organiza¬ 
tional and functional structure that will define the 
methods and procedures to manage functional and 
physical characteristics, interfaces, and documents 
of the system component. It should also include 
statements of responsibility and authority, meth¬ 
ods of control, methods of audit or verification, 
milestones, and schedules. EIA IS-649, National 
Consensus Standard for Configuration Manage¬ 
ment, and MIL-HDBK-61 can be used as plan¬ 
ning guidance. 

Configuration Item (Cl) 

A key concept that affects planning is the configu¬ 
ration item (Cl). Cl decisions will determine what 
configurations will be managed. CIs are an aggre¬ 
gation of hardware, firmware, or computer soft¬ 
ware, or any of their discrete portions, which sat¬ 
isfies an end-use function and is designated for 
separate configuration management. Any item 
required for logistic support and designated for 
separate procurement is generally identified as Cl. 
Components can be designated CIs because of 
crucial interfaces or the need to be integrated with 
operation with other components within or out¬ 
side of the system. An item can be designated Cl 
if it is developed wholly or partially with govern¬ 
ment funds, including nondevelopmental items 
(NDI) if additional development of technical data 
is required. All CIs are directly traceable to the 
WBS. 

Impact of Cl Designation 

Cl designation requires a separate configuration 
management effort for the Cl, or groupings of 


related CIs. The decision to place an item, or items, 
under formal configuration control results in: 

• Separate specifications, 

• Formal approval of changes, 

• Discrete records for configuration status 
accounting, 

• Individual design reviews and configuration 
audits, 

• Discrete identifiers and name plates, 

• Separate qualification testing, and 

• Separate operating and user manuals. 

10.2 CONFIGURATION MANAGEMENT 
STRUCTURE 

Configuration management comprises four 
interrelated efforts: 

• Identification, 

• Control, 

• Status Accounting, and 

• Audits. 

Also directly associated with configuration man¬ 
agement are data management and interface man¬ 
agement. Any configuration management planning 
effort must consider all six elements. 

Identification 

Configuration Identification consists of docu¬ 
mentation of formally approved baselines and 
specifications, including: 

• Selection of the CIs, 

• Determination of the types of configuration 
documentation required for each Cl, 
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• Documenting the functional and physical 
characteristics of each Cl, 

• Establishing interface management procedures, 
organization, and documentation, 

• Issuance of numbers and other identifiers 
associated with the system/CI configuration 
structure, including internal and external 
interfaces, and 

• Distribution of Cl identification and related 
configuration documentation. 

Configuration Documentation 

Configuration documentation is technical docu¬ 
mentation that identifies and defines the item’s 
functional and physical characteristics. It is 
developed, approved, and maintained through three 
distinct evolutionary increasing levels of detail. The 
three levels of configuration documentation form 
the three baselines and are referred to as functional, 
allocated, and product configuration documenta¬ 
tion. These provide the specific technical descrip¬ 
tion of a system or its components at any point in 
time. 

Configuration Control 

Configuration Control is the systematic proposal, 
justification, prioritization, evaluation, coordina¬ 
tion, approval or disapproval, and implementation 
of all approved changes in the configuration of a 
system/CI after formal establishment of its 
baseline. In other words, it is how a system (and 
its CIs) change control process is executed and 
managed. 

Configuration Control provides management 
visibility, ensures all factors associated with a 
proposed change are evaluated, prevents unneces¬ 
sary or marginal changes, and establishes change 
priorities. In DoD it consists primarily of a 
change process that formalizes documentation and 
provides a management structure for change 
approval. 


Change Documents Used for 
Government Controlled Baselines 

There are three types of change documents used 
to control baselines associated with government 
configuration management: Engineering Change 
Proposal, Request for Deviation, and Request for 
Waivers. 

• Engineering Change Proposals (ECP) identify 
need for a permanent configuration change. 
Upon approval of an ECP a new configuration 
is established. 

• Requests for Deviation or Waiver propose a 
temporary departure from the baseline. They 
allow for acceptance of non-conforming 
material. After acceptance of a deviation or 
waiver the documented configuration remains 
unchanged. 

Engineering Change Proposal (ECP) 

An ECP is documentation that describes and 
suggests a change to a configuration baseline. Sepa¬ 
rate ECPs are submitted for each change that has a 
distinct objective. To provide advanced notice and 
reduce paperwork. Preliminary ECPs or Advance 
Change/Study Notices can be used preparatory to 
issue of a formal ECP. Time and effort for the 
approval process can be further reduced through 
use of joint government and contractor integrated 
teams to review and edit preliminary change 
proposals. 

ECPs are identified as Class I or Class II. Class I 
changes require government approval before 
changing the configuration. These changes can 
result from problems with the baseline require¬ 
ment, safety, interfaces, operating/servicing capa¬ 
bility, preset adjustments, human interface includ¬ 
ing skill level, or training. Class I changes can also 
be used to upgrade already delivered systems to 
the new configuration through use of retrofit, mod 
kits, and the like. Class I ECPs are also used to 
change contractual provisions that do not directly 
impact the configuration baseline; for example, 
changes affecting cost, warranties, deliveries, or 
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Classification 


Justification Codes 

• Class 1 



• Class II 


D - Correction of deficiency 



S - Safety 

Types 


B - Interface 

• Preliminary 


C - Compatibility 

• Formal 


O - OPS or log support 


R - Cost reduction 

Priorities 


V - Value engineering 

• Emergency 


P - Production stoppage 

• Urgent 



• Routine 


A - Record only 


Figure 10-1. ECP Designators 


data requirements. Class I ECPs require program 
office approval, which is usually handled through 
a formal Configuration Control Board, chaired by 
the government program manager or delegated 
representative. 

Class II changes correct minor conflicts, typos, and 
other “housekeeping” changes that basically cor¬ 
rect the documentation to reflect the current con¬ 
figuration. Class II applies only if the configura¬ 
tion is not changed when the documentation is 
changed. Class II ECPs are usually handled by the 
in-plant government representative. Class II ECPs 
generally require only that the government con¬ 
curs that the change is properly classified. Under 
an initiative by the Defense Contract Management 
Command (DCMC), contractors are increasingly 
delegated the authority to make ECP classification 
decisions. 

Figure 10-1 shows the key attributes associated 
with ECPs. The preliminary ECP, mentioned in 
Figure 10-1, is a simplified version of a formal 
ECP that explains the proposed ECP, and 
establishes an approximate schedule and cost for 
the change. The expense of an ECP development 
is avoided if review of the Preliminary ECP 


indicates the change is not viable. The approach 
used for preliminary ECPs vary in their form and 
name. Both Preliminary ECPs and Advanced 
Change/Study Notices have been used to formal¬ 
ize this process, but forms tailored to specific 
programs have also been used. 

Configuration Control Board (CCB) 

A CCB is formed to review Class I ECPs for 
approval, and make a recommendation to approve 
or not approve the proposed change. The CCB 
chair, usually the program manager, makes the final 
decision. Members advise and recommend, but the 
authority for the decision rests with the chair. CCB 
membership should represent the eight primary 
functions with the addition of representation of the 
procurement office, program control (budget), and 
Configuration Control manager, who serves as the 
CCB secretariat. 

The CCB process is shown in Figure 10-2. The 
process starts with the contractor. A request to the 
contractor for an ECP or Preliminary ECP is 
necessary to initiate a government identified 
configuration change. The secretariat’s review 
process includes assuring appropriate government 
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Figure 10-2. Configuration Control Board 


contractual and engineering review is done prior 
to receipt by the CCB. 

CCB Management Philosophy 

The CCB process is a configuration control pro¬ 
cess, but it is also a contractual control process. 
Decisions made by the CCB chair affects the con¬ 
tractual agreement and program baseline as well 
as the configuration baseline. Concerns over con¬ 
tractual policy, program schedule, and budget can 
easily come into conflict with concerns relating to 
configuration management, technical issues, and 
technical activity scheduling. The CCB technical 
membership and CCB secretariat is responsible to 
provide a clear view of the technical need and the 
impact of alternate solutions to these conflicts. The 
CCB secretariat is further responsible to see that 
the CCB is fully informed and prepared, including 
ensuring that: 

• A government/contractor engineering working 
group has analyzed the ECP and supporting data, 
prepared comments for CCB consideration, and 
is available to support the CCB; 


• All pertinent information is available for review; 

• The ECP has been reviewed by appropriate 
functional activities; and 

• Issues have been identified and addressed. 

CCB Documentation 

Once the CCB chair makes a decision concerning 
an ECP, the CCB issues a Configuration Control 
Board Directive that distributes the decision and 
identifies key information relating to the imple¬ 
mentation of the change: 

• Implementation plan (who does what when); 

• Contracts affected (prime and secondary); 

• Dates of incorporation into contracts; 

• Documentation affected (drawings, specifica¬ 
tions, technical manuals, etc.), associated cost, 
and schedule completion date; and 
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• Identification of any orders or directives needed 
to be drafted and issued. 

Request for Deviation or Waiver 

A deviation is a specific written authorization, 
granted prior to manufacture of an item, to depart 
from a performance or design requirement for a 
specific number of units or a specific period of 
time. 

A waiver is a written authorization to accept a Cl 
that departs from specified requirements, but is 
suitable for use “as is” or after repair. 

Requests for deviation and waivers relate to a tem¬ 
porary baseline departure that can affect system 
design and/or performance. The baseline remains 
unchanged and the government makes a determi¬ 
nation whether the alternative “non-conforming” 
configuration results in an acceptable substitute. 
Acceptable substitute usually implies that there will 
be no impact on support elements, systems affected 
can operate effectively, and no follow-up or cor¬ 
rection is required. The Federal Acquisition Regu¬ 
lations (FAR) requires “consideration” on govern¬ 
ment contracts when the Government accepts a 
“non-conforming” unit. 

The distinction between Request for Deviation and 
Request for a Waiver is that a deviation is used 
before final assembly of the affected unit, and a 
waiver is used after final assembly or acceptance 
testing of the affected unit. 

Status Accounting 

Configuration Status Accounting is the recording 
and reporting of the information that is needed to 
manage the configuration effectively, including: 

• A listing of the approved configuration docu¬ 
mentation, 

• The status of proposed changes, waivers and 
deviations to the configuration identification, 

• The implementation status of approved changes, 
and 


• The configuration of all units, including those 
in the operational inventory. 

Purpose of Configuration Status Accounting 

Configuration Status Accounting provides infor¬ 
mation required for configuration management by: 

• Collecting and recording data concerning: 

- Baseline configurations, 

- Proposed changes, and 

- Approved changes. 

• Disseminating information concerning: 

- Approved configurations, 

- Status and impact of proposed changes, 

- Requirements, schedules, impact and 
status of approved changes, and 

- Current configurations of delivered items. 

Audits 

Configuration Audits are used to verify a system 
and its components’ conformance to their configu¬ 
ration documentation. Audits are key milestones 
in the development of the system and do not stand 
alone. The next chapter will show how they fit in 
the overall process of assessing design maturity. 

Functional Configuration Audits (FCA) and the 
System Verification Review (SVR) are performed 
in the Production Readiness and LRIP stage of 
the Production and Development Phase. FCA 
is used to verify that actual performance of the 
configuration item meets specification require¬ 
ments. The SVR serves as system-level audit after 
FCAs have been conducted. 

The Physical Configuration Audit (PCA) is nor¬ 
mally held during Rate Production and Develop¬ 
ment stage as a formal examination of a pro¬ 
duction representative unit against the draft tech¬ 
nical data package (product baseline documenta¬ 
tion). 

Most audits, whether FCA or PCA, are today 
approached as a series of “rolling” reviews in which 
items are progressively audited as they are pro¬ 
duced such that the final FCA or PCA becomes 
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significantly less oppressive and disruptive to the 
normal flow of program development. 

10.3 INTERFACE MANAGEMENT 

Interface Management consists of identifying the 
interfaces, establishing working groups to manage 
the interfaces, and the group’s development of in¬ 
terface control documentation. Interface Manage¬ 
ment identifies, develops, and maintains the exter¬ 
nal and internal interfaces necessary for system 
operation. It supports the configuration manage¬ 
ment effort by ensuring that configuration 
decisions are made with full understanding of their 
impact outside of the area of the change. 

Interface Identification 

An interface is a functional, physical, electrical, 
electronic, mechanical, hydraulic, pneumatic, op¬ 
tical, software, or similar characteristic required 
to exist at a common boundary between two or 
more systems, products, or components. Normally, 
in a contractual relationship the procuring agency 
identifies external interfaces, sets requirements for 
integrated teams, and provides appropriate person¬ 
nel for the teams. The contracted design agent or 
manufacturer manages internal interfaces; plans, 
organizes, and leads design integrated teams; main¬ 
tains internal and external interface requirements; 
and controls interfaces to ensure accountability and 
timely dissemination of changes. 

Interface Control Working Group (ICWG) 

The ICWG is the traditional forum to establish 
official communications link between those 
responsible for the design of interfacing systems 
or components. Within the IPPD framework 
ICWGs can be integrated teams that establish link¬ 
age between interfacing design IPTs, or could be 
integrated into a system-level engineering work¬ 
ing group. Membership of ICWGs or comparable 
integrated teams should include membership from 
each contractor, significant vendors, and partici¬ 
pating government agencies. The procuring 


program office (external and selected top-level 
interfaces) or prime contractor (internal interfaces) 
generally designates the chair. 

Interface Control Documentation (ICD) 

Interface Control Documentation includes Inter¬ 
face Control Drawings, Interface Requirements 
Specifications, and other documentation that 
depicts physical and functional interfaces of related 
or co-functioning systems or components. ICD is 
the product of ICWGs or comparable integrated 
teams, and their puipose is to establish and main¬ 
tain compatibility between interfacing systems or 
components. 

Open Systems Interface Standards 

To minimize the impact of unique interface 
designs, improve interoperability, maximize the 
use of commercial components, and improve the 
capacity for future upgrade, an open-systems ap¬ 
proach should be a significant paid of interface 
control planning. The open-systems approach in¬ 
volves selecting industry-recognized specifications 
and standards to define system internal and exter¬ 
nal interfaces. An open system is characterized by: 

• Increased use of functional partitioning and 
modular design to enhance flexibility of 
component choices without impact on inter¬ 
faces, 

• Use of well-defined, widely used, non-propri¬ 
etary interfaces or protocols based on standards 
developed or adopted by industry recognized 
standards institutions or professional societies, 
and 

• Explicit provision for expansion or upgrading 
through the incorporation of additional or 
higher performance elements with minimal 
impact on the system. 

DoD mandatory guidance for information tech¬ 
nology standards is in the Joint Technical Archi¬ 
tecture. 
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10.4 DATA MANAGEMENT 

Data management documents and maintains the 
database reflecting system life cycle decisions, 
methods, feedback, metrics, and configuration 
control. It directly supports the configuration sta¬ 
tus accounting process. Data Management governs 
and controls the selection, generation, preparation, 
acquisition, and use of data imposed on contractors. 

Data Required By Contract 

Data is defined as recorded information, regard¬ 
less of form or characteristic, and includes all the 
administrative, management, financial, scientific, 
engineering, and logistics information and docu¬ 
mentation required for delivery from the contrac¬ 
tor. Contractually required data is classified as one 
of three types: 

• Type I: Technical data 

• Type II: Non-technical data 

• Type III: One-time use data (technical or non¬ 
technical) 

Data is acquired for two basic puiposes: 

• Information feedback from the contractor for 
program management control, and 

• Decision making information needed to 
manage, operate, and support the system (e.g., 
specifications, technical manuals, engineering 
drawings, etc.). 

Data analysis and management is expensive and 
time consuming. Present DoD philosophy requires 
that the contractor manage and maintain signifi¬ 
cant portions of the technical data, including the 
Technical Data Package (TDP). Note that this does 
not mean the government isn’t paying for its 
development or shouldn’t receive a copy for post¬ 
delivery use. Minimize the TDP cost by request¬ 
ing the contractor’s format (for example, accept¬ 
ing the same drawings they use for production), 
and asking only for details on items developed with 
government funds. 


Data Call for Government Contracts 

As part of the development of an Invitation for Bid 
or Request for Proposals, the program office is¬ 
sues a letter that describes the planned procure¬ 
ment and asks integrated team leaders and effected 
functional managers to identify and justify their 
data requirements for that contract. A description 
of each data item needed is then developed by the 
affected teams or functional offices, and reviewed 
by the program office. Data Item Descriptions, 
located in the Acquisition Management Systems 
Data List (AMSDL) (see Chapter 8) can be used 
for guidance in developing these descriptions. 

Concurrent with the DoD policy on specifications 
and standards, there is a trend to avoid use of stan¬ 
dard Data Item Descriptions on contracts, and 
specify the data item with a unique tailored data 
description referenced in the Contract Data 
Requirements List. 

10.5 SUMMARY POINTS 

• Configuration management is essential to con¬ 
trol the system design throughout the life cycle. 

• Use of integrated teams in an IPPD environ¬ 
ment is necessary for disciplined configuration 
management of complex systems. 

• Technical data management is essential to trace 
decisions and changes and to document designs, 
processes and procedures. 

• Interface management is essential to ensure that 
system elements are compatible in terms of 
form, fit, and function. 

• Three configuration baselines are managed: 

- Functional (System level) 

- Allocated (Design To) 

- Product (Build To/As Built) 

Configuration management is a shared responsi¬ 
bility between the government and the contractor. 
Contract manager (CM) key elements are Identifi¬ 
cation, Control, Status Accounting, and Audits. 
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11.1 PROGRESS MEASUREMENT 

The Systems Engineer measures design progress 
and maturity by assessing its development at key 
event-driven points in the development schedule. 
The design is compared to pre-established exit 
criteria for the particular event to determine if the 
appropriate level of maturity has been achieved. 
These key events are generally known as Technical 
Reviews and Audits. 

A system in development proceeds through a 
sequence of stages as it proceeds from concept to 
finished product. These are referred to as “levels 
of development.” Technical Reviews are done after 
each level of development to check design matu¬ 
rity, review technical risk, and determines whether 
to proceed to the next level of development. Tech¬ 
nical Reviews reduce program risk and ease the 
transition to production by: 

• Assessing the maturity of the design/develop¬ 
ment effort, 

• Clarifying design requirements, 

• Challenging the design and related processes, 

• Checking proposed design configuration 
against technical requirements, customer needs, 
and system requirements, 

• Evaluating the system configuration at different 
stages, 

• Providing a forum for communication, coordi¬ 
nation, and integration across all disciplines and 
IPTs, 


• Establishing a common configuration baseline 
from which to proceed to the next level of 
design, and 

• Recording design decision rationale in the 
decision database. 

Formal technical reviews are preceded by a series 
of technical interchange meetings where issues, 
problems and concerns are surfaced and addressed. 
The formal technical review is NOT the place for 
problem solving, but to verify problem solving has 
been done; it is a process rather than an event! 

Planning 

Planning for Technical Reviews must be extensive 
and up-front-and-early. Important considerations 
for planning include the following: 

• Timely and effective attention and visibility into 
the activities preparing for the review, 

• Identification and allocation of resources 
necessary to accomplish the total review effort, 

• Tailoring consistent with program risk levels, 

• Scheduling consistent with availability of 
appropriate data, 

• Establishing event-driven entry and exit criteria, 

• Where appropriate, conduct of incremental 
reviews, 

• Implementation by IPTs, 
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• Review of all system functions, and 

• Confirmation that all system elements are 
integrated and balanced. 

The maturity of enabling products are reviewed 
with their associated end product. Reviews should 
consider the testability, producibility, training, and 
supportability for the system, subsystem or 
configuration item being addressed. 

The depth of the review is a function of the com¬ 
plexity of the system, subsystem, or configuration 
item being reviewed. Where design is pushing 
state-of-the-art technology the review will require 
a greater depth than if it is for a commercial off- 
the-shelf item. Items, which are complex or an 
application of new technology, will require a more 
detailed scrutiny. 


Planning Tip: Develop a check list of pre-review, 
review, and post-review activities required. De¬ 
velop check lists for exit criteria and required level 
of detail in design documentation. Include key 
questions to be answered and what information 
must be available to facilitate the review process. 
Figure 11-1 shows the review process with key 
activities identified. 


11.2 TECHNICAL REVIEWS 

Technical reviews are conducted at both the sys¬ 
tem level and at lower levels (e.g., sub-system). 
This discussion will focus on the primary system- 
level reviews. Lower-level reviews may be thought 
of as events that support and prepare for the sys¬ 
tem-level events. The names used in reference to 
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reviews is unimportant; however, it is important 
that reviews be held at appropriate points in pro¬ 
gram development and that both the contractor and 
government have common expectations regarding 
the content and outcomes. 

Conducting Reviews 

Reviews are event-driven, meaning that they are 
to be conducted when the progress of the product 
under development merits review. Forcing a review 
(simply based on the fact that a schedule devel¬ 
oped earlier) projected the review at a point in time 
will jeopardize the review’s legitimacy. Do the 
work ahead of the review event. Use the review 
event as a confirmation of completed effort. The 
data necessary to determine if the exit criteria are 
satisfied should be distributed, analyzed, and 
analysis coordinated prior to the review. The type 
of information needed for a technical review 
would include: specifications, drawings, manuals, 


schedules, design and test data, trade studies, risk 
analysis, effectiveness analyses, mock-ups, bread¬ 
boards, in-process and finished hardware, test 
methods, technical plans (Manufacturing, Test, 
Support, Training), and trend (metrics) data. Re¬ 
views should be brief and follow a prepared agenda 
based on the pre-review analysis and assessment 
of where attention is needed. 

Only designated participants should personally 
attend. These individuals should be those that were 
involved in the preparatory work for the review 
and members of the IPTs responsible for meeting 
the event exit criteria. Participants should include 
representation from all appropriate government 
activities, contractor, subcontractors, vendors and 
suppliers. 

A review is the confirmation of a process. New 
items should not come up at the review. If signifi¬ 
cant items do emerge, it’s a clear sign the review is 
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being held prematurely, and project risk has just 
increased significantly. A poorly orchestrated and 
performed technical review is a significant 
indicator of management problems. 

Action items resulting from the review are docu¬ 
mented and tracked. These items, identified by 
specific nomenclature and due dates, are prepared 
and distributed as soon as possible after the review. 
The action taken is tracked and results distributed 
as items are completed. 

Phasing of Technical Reviews 

As a system progresses through design and devel¬ 
opment, it typically passes from a given level of 
development to another, more advanced level of 
development. For example, a typical system will 
pass from a stage where only the requirements are 
known, to another stage where a conceptual 
solution has been defined. Or it may pass from a 
stage where the design requirements for the 
primary subsystems are formalized, to a stage 
where the physical design solutions for those 
requirements are defined. (See Figure 11-2.) 


These stages are the “levels of development” re¬ 
ferred to in this chapter. System-level technical 
reviews arc generally timed to correspond to the 
transition from one level of development to an¬ 
other. The technical review is the event at which 
the technical manager verifies that the technical 
maturity of the system or item under review is suf¬ 
ficient to justify passage into the subsequent phase 
of development, with the concomitant commitment 
of resources required. 

As the system or product progresses through 
development, the focus of technical assessment 
takes different forms. Early in the process, the pri¬ 
mary focus is on defining the requirements on 
which subsequent design and development activi¬ 
ties will be based. Similarly, technical reviews 
conducted during the early stages of develop¬ 
ment are almost always focused on ensuring that 
the top-level concepts and system definitions 
reflect the requirements of the user. Once system- 
level definition is complete, the focus turns to de¬ 
sign at sub-system levels and below. Technical re¬ 
views during these stages are typically design re¬ 
views that establish design requirements and then 
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verify that physical solutions are consistent with 
those requirements. In the final stages of develop¬ 
ment, technical reviews and audits are conducted 
to verify that the products produced meet the re¬ 
quirements on which the development is based. 
Figure 11-3 summarizes the typical schedule of 
system-level reviews by type and focus. 

Another issue associated with technical reviews, 
as well as other key events normally associated 
with executing the systems engineering process, 
is when those events generally occur relative to 
the phases of the DoD acquisition life-cycle 
process. The timing of these events will vary some¬ 
what from program to program, based upon the 
explicit and unique needs of the situation; how¬ 
ever, Figure 11-4 shows a generalized concept of 
how the technical reviews normal to systems 
engineering might occur relative to the acquisition 
life-cycle phases. 


Specific system-level technical reviews are known 
by many different names, and different engi¬ 
neering standards and documents often use differ¬ 
ent nomenclature when referring to the same 
review. The names used to refer to technical 
reviews are unimportant; however, it is important 
to have a grasp of the schedule of reviews that is 
normal to system development and to have an 
understanding of what is the focus and puipose of 
those reviews. The following paragraphs outline a 
schedule of reviews that is complete in terms of 
assessing technical progress from concept through 
production. The names used were chosen because 
they seemed to be descriptive of the focus of the 
activity. Of course, the array of reviews and the 
focus of individual reviews is to be tailored to the 
specific needs of the program under development, 
so not all programs should plan on conducting all 
of the following reviews. 
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Alternative Systems Review (ASR) 

After the concept studies are complete a preferred 
system concept is identified. The associated draft 
System Work Breakdown Structure, preliminary 
functional baseline, and draft system specification 
are reviewed to determine feasibility and risk. 
Technology dependencies are reviewed to ascer¬ 
tain the level of technology risk associated with 
the proposed concepts. This review is conducted 
late during the Concept Exploration stage of the 
Concept and Technology Development Phase of 
the acquisition process to verify that the preferred 
system concept: 

• Provides a cost-effective, operationally-effective 
and suitable solution to identified needs, 

• Meets established affordability criteria, and 

• Can be developed to provide a timely solution 
to the need at an acceptable level of risk. 

The findings of this review are a significant input 
to decision review conducted after Concept 
Exploration to determine where the system should 
enter in the life-cycle process to continue devel¬ 
opment. This determination is largely based on 
technology and system development maturity. 

It is important to understand that the path of the 
system through the life-cycle process will be 
different for systems of different maturities. Con¬ 
sequently, the decision as whether or not to conduct 
the technical reviews that are briefly described in 
the following paragraphs is dependent on the extent 
of design and development required to bring the 
system to a level of maturity that justifies producing 
and fielding it. 

System Requirements Review (SRR) 

If a system architecture system must be developed 
and a top-down design elaborated, the system will 
pass through a number of well-defined levels of 
development, and that being the case, a well- 
planned schedule of technical reviews is impera¬ 
tive. The Component Advanced Development stage 
(the second stage of Concept and Technology 


Development in the revised acquisition life-cycle 
process) is the stage during which system-level ar¬ 
chitectures are defined and any necessary advanced 
development required to assess and control tech¬ 
nical risk is conducted. As the system passes into 
the acquisition process, i.e., passes a Milestone B 
and enters System Development and Demonstra¬ 
tion, it is appropriate to conduct a SRR. The SRR 
is intended to confirm that the user’s requirements 
have been translated into system specific techni¬ 
cal requirements, that critical technologies are iden¬ 
tified and required technology demonstrations are 
planned, and that risks are well understood and 
mitigation plans are in place. The draft system 
specification is verified to reflect the operational 
requirements. 

All relevant documentation should be reviewed, 
including: 

• System Operational Requirements, 

• Draft System Specification and any initial draft 
Performance Item Specifications, 

• Functional Analysis (top level block diagrams), 

• Feasibility Analysis (results of technology 
assessments and trade studies to justify system 
design approach), 

• System Maintenance Concept, 

• Significant system design criteria (reliability, 
maintainability, logistics requirements, etc.), 

• System Engineering Planning, 

• Test and Evaluation Master Plan, 

• Draft top-level Technical Performance Measure¬ 
ment, and 

• System design documentation (layout drawings, 
conceptual design drawings, selected supplier 
components data, etc.). 

The SRR confirms that the system-level require¬ 
ments are sufficiently well understood to permit 
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the developer (contractor) to establish an initial sys¬ 
tem level functional baseline. Once that baseline is 
established, the effort begins to define the function¬ 
al, performance, and physical attributes of the items 
below system level and to allocate them to the 
physical elements that will perform the functions. 

System Functional Review (SFR) 

The process of defining the items or elements 
below system level involves substantial engineer¬ 
ing effort. This design activity is accompanied by 
analysis, trade studies, modeling and simulation, 
as well as continuous developmental testing to 
achieve an optimum definition of the major ele¬ 
ments that make up the system, with associated 
functionality and performance requirements. This 
activity results in two major systems engineering 
products: the final version of the system perfor¬ 
mance specification and draft versions of the 
performance specifications, which describe the 
items below system level (item performance speci¬ 
fications). These documents, in turn, define the 
system functional baseline and the draft allocated 
baseline. As this activity is completed, the system 
has passed from the level of a concept to a well- 
defined system design, and, as such, it is appropri¬ 
ate to conduct another in the series of technical 
reviews. 

The SFR will typically include the tasks listed 
below. Most importantly, the system technical 
description (Functional Baseline) must be ap¬ 
proved as the governing technical requirement 
before proceeding to further technical development. 
This sets the stage for engineering design and 
development at the lower levels in the system 
architecture. The government, as the customer, 
will normally take control of and manage the 
system functional baseline following successful 
completion of the SFR. 

The review should include assessment of the fol¬ 
lowing items. More complete lists are found in 
standards and texts on the subject. 

• Verification that the system specification 
reflects requirements that will meet user 
expectations. 


• Functional Analysis and Allocation of require¬ 
ments to items below system level, 

• Draft Item Performance and some Item Detail 
Specifications, 

• Design data defining the overall system, 

• Verification that the risks associated with the 
system design are at acceptable levels for 
engineering development, 

• Verification that the design selections have been 
optimized through appropriate trade study 
analyses, 

• Supporting analyses, e.g., logistics, human sys¬ 
tems integration, etc., and plans are identified 
and complete where appropriate, 

• Technical Performance Measurement data and 
analysis, and 

• Plans for evolutionary design and development 
are in place and that the system design is 
modular and open. 

Following the SFR, work proceeds to complete the 
definition of the design of the items below system 
level, in terms of function, performance, interface 
requirements for each item. These definitions are 
typically captured in item performance specifica¬ 
tions, sometimes referred to as prime item devel¬ 
opment specifications. As these documents are 
finalized, reviews will normally be held to verify 
that the design requirements at the item level reflect 
the set of requirements that will result in an 
acceptable detailed design, because all design work 
from the item level to the lowest level in the system 
will be based on the requirements agreed upon at 
the item level. The establishment of a set of final 
item-level design requirements represents the defi¬ 
nition of the allocated baseline for the system. 
There are two primary reviews normally associ¬ 
ated with this event: the Software Specification 
Review (SSR), and the Preliminary Design Review 
(PDR). 
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Software Specification Review (SSR) 

As system design decisions are made, typically 
some functions are allocated to hardware items, 
while others are allocated to software. A separate 
specification is developed for software items to 
describe the functions, performance, interfaces and 
other information that will guide the design and 
development of software items. In preparation for 
the system-level PDR, the system software 
specification is reviewed prior to establishing the 
Allocated Baseline. The review includes: 

• Review and evaluate the maturity of software 
requirements, 

• Validation that the software requirements speci¬ 
fication and the interface requirements speci¬ 
fication reflect the system-level requirements 
allocated to software, 

• Evaluation of computer hardware and software 
compatibility, 

• Evaluation of human interfaces, controls, and 
displays 

• Assurance that software-related risks have been 
identified and mitigation plans established, 

• Validation that software designs are consistent 
with the Operations Concept Document, 

• Plans for testing, and 

• Review of preliminary manuals. 

Preliminary Design Review (PDR) 

Using the Functional Baseline, especially the 
System Specification, as a governing requirement, 
a preliminary design is expressed in terms of design 
requirements for subsystems and configuration 
items. This preliminary design sets forth the func¬ 
tions, performance, and interface requirements that 
will govern design of the items below system level. 
Following the PDR, this preliminary design (Allo¬ 
cated Baseline) will be put under formal config¬ 
uration control [usually] by the contractor. The 


Item Performance Specifications, including the 
system software specification, which form the 
core of the Allocated Baseline, will be confirmed 
to represent a design that meets the System 
Specification. 

This review is performed during the System 
Development and Demonstration phase. Reviews 
are held for configuration items (CIs), or groups 
of related CIs, prior to a system-level PDR. Item 
Performance Specifications are put under configu¬ 
ration control (Current DoD practice is for con¬ 
tractors to maintain configuration control over Item 
Performance Specifications, while the government 
exercises requirements control at the system 
level). At a minimum, the review should include 
assessment of the following items: 

• Item Performance Specifications, 

• Draft Item Detail, Process, and Material 
Specifications, 

• Design data defining major subsystems, 
equipment, software, and other system 
elements, 

• Analyses, reports, “ility” analyses, trade stud¬ 
ies, logistics support analysis data, and design 
documentation, 

• Technical Performance Measurement data and 
analysis, 

• Engineering breadboards, laboratory models, 
test models, mockups, and prototypes used to 
support the design, and 

• Supplier data describing specific components. 

[Rough Rule of Thumb: -15% of production draw¬ 
ings are released by PDR. This rule is anecdotal 
and only guidance relating to an “average” defense 
hardware program.] 

Critical Design Review (CDR) 

Before starting to build the production line there 
needs to be verification and formalization of the 
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mutual understanding of the details of the item 
being produced. Performed during the System 
Development and Demonstration phase, this re¬ 
view evaluates the draft Production Baseline 
(“Build To” documentation) to determine if the 
system design documentation (Product Baseline, 
including Item Detail Specs, Material Specs, Pro¬ 
cess Specs) is satisfactory to start initial manufac¬ 
turing. This review includes the evaluation of all 
CIs. It includes a series of reviews conducted for 
each hardware Cl before release of design to fab¬ 
rication, and each computer software Cl before 
final coding and testing. Additionally, test plans 
are reviewed to assess if test efforts are develop¬ 
ing sufficiently to indicate the Test Readiness 
Review will be successful. The approved detail 
design serves as the basis for final production 
planning and initiates the development of final 
software code. 

[Rough Rule of Thumb: At CDR the design should 
be at least 85% complete. Many programs use 
drawing release as a metric for measuring design 
completion. This rule is anecdotal and only guid¬ 
ance relating to an “average” defense hardware 
program.] 

Test Readiness Review (TRR) 

Typically performed during the System Demon¬ 
stration stage of the System Development and 
Demonstration phase (after CDR), the TRR as¬ 
sesses test objectives, procedures, and resources 
testing coordination. Originally developed as a 
software Cl review, this review is increasingly 
applied to both hardware and software items. The 
TRR determines the completeness of test proce¬ 
dures and their compliance with test plans and 
descriptions. Completion coincides with the 
initiation of formal Cl testing. 

Production Readiness Reviews (PRR) 

Performed incrementally during the System 
Development and Demonstration and during the 
Production Readiness stage of the Production and 
Deployment phase, this series of reviews is held 
to determine if production preparation for the sys¬ 
tem, subsystems, and configuration items is com¬ 


plete, comprehensive, and coordinated. PRRs are 
necessary to determine the readiness for produc¬ 
tion prior to executing a production go-ahead 
decision. They will formally examine the pro- 
ducibility of the production design, the control over 
the projected production processes, and adequacy 
of resources necessary to execute production. 
Manufacturing risk is evaluated in relationship to 
product and manufacturing process performance, 
cost, and schedule. These reviews support acqui¬ 
sition decisions to proceed to Low-Rate Initial 
Production (LRIP) or Full-Rate Production. 

Functional Configuration Audit/ System 
Verification Review (FCA)/(SVR) 

This series of audits and the consolidating SVR 
re-examines and verifies the customer’s needs, and 
the relationship of these needs to the system and 
subsystem technical performance descriptions 
(Functional and Allocated Baselines). They deter¬ 
mine if the system produced (including produc¬ 
tion representative prototypes or LRIP units) is 
capable of meeting the technical performance 
requirements established in the specifications, test 
plans, etc. The FCA verifies that all requirements 
established in the specifications, associated test 
plans, and related documents have been tested and 
that the item has passed the tests, or collective 
action has been initiated. The technical assessments 
and decisions that arc made in SVR will be pre¬ 
sented to support the full-rate production go-ahead 
decision. Among the issues addressed: 

• Readiness issues for continuing design, continu¬ 
ing verifications, production, training, deploy¬ 
ment, operations, support, and disposal have 
been resolved, 

• Verification is comprehensive and complete, 

• Configuration audits, including completion of all 
change actions, have been completed for all CIs, 

• Risk management planning has been updated 
for production, 

• Systems Engineering planning is updated for 
production, and 


107 



Systems Engineering Fundamentals 


Chapter 11 


• Critical achievements, success criteria and 
metrics have been established for production. 

Physical Configuration Audit (PCA) 

After full-rate production has been approved, fol- 
low-on independent verification (FOT&E) has 
identified the changes the user requires, and those 
changes have been corrected on the baseline docu¬ 
ments and the production line, then it is time to 
assure that the product and the product baseline 
documentation arc consistent. The PCA will for¬ 
malize the Product Baseline, including specifica¬ 
tions and the technical data package, so that future 
changes can only be made through full configura¬ 
tion management procedures. Fundamentally, the 
PCA verifies the product (as built) is consistent 
with the Technical Data Package which describes 
the Product Baseline. The final PCA confirms: 

• The subsystem and Cl PCAs have been 
successfully completed, 

• The integrated decision database is valid and 
represents the product, 

• All items have been baselined, 

• Changes to previous baselines have been 
completed, 

• Testing deficiencies have been resolved and 
appropriate changes implemented, and 

• System processes are current and can be 
executed. 

The PCA is a configuration management activity 
and is conducted following procedures established 
in the Configuration Management Plan. 

11.3 TAILORING 

The reviews described above are based on a 
complex system development project requiring 
significant technical evaluation. There are also 


cases where system technical maturity is more 
advanced than normal for the phase, for example, 
where a previous program or an Advanced Tech¬ 
nical Concept Demonstration (ACTD) has pro¬ 
vided a significant level of technical development 
applicable to the current program. In some cases 
this will precipitate the merging or even elimina¬ 
tion of acquisition phases. This does not justify 
elimination of the technical management activi¬ 
ties grouped under the general heading of systems 
analysis and control, nor does it relieve the 
government program manager of the responsibil¬ 
ity to see that these disciplines are enforced. It does, 
however, highlight the need for flexibility and 
tailoring to the specific needs of the program under 
development. 

For example, a DoD acquisition strategy that pro¬ 
poses that a system proceed directly into the dem¬ 
onstration stage may skip a stage of the complete 
acquisition process, but it must not skip the for¬ 
mulation of an appropriate Functional Baseline and 
the equivalent of an SFR to support the develop¬ 
ment. Nor should it skip the formulation of the 
Allocated Baseline and the equivalent of a PDR, 
and the formulation of the Product Baseline and 
the equivalent of a CDR. Baselines must be devel¬ 
oped sequentially because they document differ¬ 
ent levels of design requirements and must build 
on each other. However, the assessment of design 
and development maturity can be tailored as ap¬ 
propriate for the particular system. Tailored efforts 
still have to deal with the problem of determining 
when the design maturity should be assessed, and 
how these assessments will support the formula¬ 
tion and control of baselines, which document the 
design requirements as the system matures. 

In tailoring efforts, be extremely careful determin¬ 
ing the level of system complexity. The system 
integration effort, the development of a single 
advanced technology or complex sub-component, 
or the need for intensive software development may 
be sufficient to establish the total system as a com¬ 
plex project, even though it appears simple because 
most subsystems are simple or off-the-shelf. 
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11.4 SUMMARY POINTS 

• Each level of product development is evaluated 
and progress is controlled by specification de¬ 
velopment (System, Item Performance, Item 
Detail, Process, and Material specifications) and 
technical reviews and audits (ASR, SRR, SDR, 
SSR, PDR, CDR, TRR, PRR, FCA, SVR, 
PC A). 

• Technical reviews assess development maturity, 
risk, and cost/schedule effectiveness to deter¬ 
mine readiness to proceed. 

• Reviews must be planned, managed, and 
followed up to be effective as an analysis and 
control tool. 


• As the system progresses through the develop¬ 
ment effort, the nature of design reviews and 
audits will parallel the technical effort. Initially 
they will focus on requirements and functions, 
and later become very product focused. 

• After system level reviews establish the Func¬ 
tional Baseline, technical reviews tend to be 
subsystem and Cl focused until late in devel¬ 
opment when the focus again turns to the sys¬ 
tem level to determine the system’s readiness 
for production. 
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12.1 MAKING CHOICES 

Trade Studies are a formal decision making meth¬ 
odology used by integrated teams to make choices 
and resolve conflicts during the systems engineer¬ 
ing process. Good trade study analyses demand 
the participation of the integrated team; otherwise, 
the solution reached may be based on unwarranted 
assumptions or may reflect the omission of 
important data. 

Trade studies identify desirable and practical 
alternatives among requirements, technical objec¬ 
tives, design, program schedule, functional and 
performance requirements, and life-cycle costs are 
identified and conducted. Choices are then made 
using a defined set of criteria. Trade studies are 
defined, conducted, and documented at the vari¬ 
ous levels of the functional or physical architec¬ 
ture in enough detail to support decision making 
and lead to a balanced system solution. The level 
of detail of any trade study needs to be commen¬ 
surate with cost, schedule, performance, and risk 
impacts. 

Both formal and informal trade studies are con¬ 
ducted in any systems engineering activity. For¬ 
mal trade studies tend to be those that will be used 
in formal decision forums, e.g., milestone deci¬ 
sions. These are typically well documented and 
become a part of the decision database normal to 
systems development. On the other hand, engineer¬ 
ing choices at every level involve trade-offs and 
decisions that parallel the trade study process. Most 
of these less-formal studies are documented in 
summary detail only, but they are important in that 
they define the design as it evolves. 


Systems Engineering Process 
and Trade Studies 

Trade studies are required to support decisions 
throughout the systems engineering process. Dur¬ 
ing requirements analysis, requirements are bal¬ 
anced against other requirements or constraints, 
including cost. Requirements analysis trade stud¬ 
ies examine and analyze alternative performance 
and functional requirements to resolve conflicts 
and satisfy customer needs. 

During functional analysis and allocation, func¬ 
tions are balanced with interface requirements, 
dictated equipment, functional partitioning, 
requirements flowdown, and configuration items 
designation considerations. Trade studies are 
conducted within and across functions to: 

• Support functional analyses and allocation of 
performance requirements and design con¬ 
straints, 

• Define a preferred set of performance require¬ 
ments satisfying identified functional interfaces, 

• Deteimine performance requirements for lower- 
level functions when higher-level performance 
and functional requirements can not be readily 
resolved to the lower-level, and 

• Evaluate alternative functional architectures. 

During design synthesis, trade studies are used to 
evaluate alternative solutions to optimize cost, 
schedule, performance, and risk. Trade studies are 
conducted during synthesis to: 
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• Support decisions for new product and process 
developments versus non-developmental 
products and processes; 

• Establish system, subsystem, and component 
configurations; 

• Assist in selecting system concepts, designs, 
and solutions (including people, parts, and 
materials availability); 

• Support materials selection and make-or-buy, 
process, rate, and location decisions; 

• Examine proposed changes; 

• Examine alternative technologies to satisfy 
functional or design requirements including 
alternatives for moderate- to high- risk 
technologies; 

• Evaluate environmental and cost impacts of 
materials and processes; 

• Evaluate alternative physical architectures to 
select preferred products and processes; and 

• Select standard components, techniques, 
services, and facilities that reduce system life- 
cycle cost and meet system effectiveness 
requirements. 

During early program phases, for example, during 
Concept Exploration and functional baseline 
development, trade studies are used to examine 
alternative system-level concepts and scenarios to 
help establish the system configuration. During 
later phases, trade studies are used to examine 
lower-level system segments, subsystems, and end 
items to assist in selecting component part designs. 
Performance, cost, safety, reliability, risk, and other 
effectiveness measures must be traded against each 
other and against physical characteristics. 


12.2 TRADE STUDY BASICS 

Trade studies (hade-off analyses) are processes that 
examine viable alternatives to determine which is 


preferred. It is important that there be criteria 
established that are acceptable to ah members of 
the integrated team as a basis for a decision. In 
addition, there must be an agreed-upon approach 
to measuring alternatives against the criteria. If 
these principles are followed, the hade study should 
produce decisions that are rational, objective, and 
repeatable. Finally, trade study results must be such 
that they can be easily communicated to custom¬ 
ers and decision makers. If the results of a trade 
study are too complex to communicate with ease, 
it is unlikely that the process will result in timely 
decisions. 

Trade Study Process 

As shown by Figure 12-1, the process of trade-off 
analysis consists of defining the problem, bound¬ 
ing the problem, establishing a hade-off method¬ 
ology (to include the establishment of decision 
criteria), selecting alternative solutions, determin¬ 
ing the key characteristics of each alternative, 
evaluating the alternatives, and choosing a solution: 

• Defining the problem entails developing a 
problem statement including any constraints. 
Problem definition should be done with exheme 
care. After all, if you don’t have the right 
problem, you won’t get the right answer. 

• Bounding and understanding the problem 
requires identification of system requirements 
that apply to the study. 

• Conflicts between desired characteristics of the 
product or process being studied, and the 
limitations of available data. Available databases 
should be identified that can provide relevant, 
historical “actual” information to support 
evaluation decisions. 

• Establishing the methodology includes choos¬ 
ing the mathematical method of comparison, 
developing and quantifying the criteria used for 
comparison, and determining weighting factors 
(if any). Use of appropriate models and meth¬ 
odology will dictate the rationality, objectivity, 
and repeatability of the study. Experience has 
shown that this step can be easily abused 
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Figure 12-1. Trade Study Process 


through both ignorance and design. To the ex¬ 
tent possible the chosen methodology should 
compare alternatives based on true value to the 
customer and developer. Trade-off relationships 
should be relevant and rational. Choice of util¬ 
ity or weights should answer the question, “what 
is the actual value of the increased performance, 
based on what rationale?” 

• Selecting alternative solutions requires identi¬ 
fication of all the potential ways of solving the 
problem and selecting those that appear viable. 
The number of alternatives can drive the cost 
of analysis, so alternatives should normally be 
limited to clearly viable choices. 


• Determining the key characteristics entails 
deriving the data required by the study 
methodology for each alternative. 

• Evaluating the alternatives is the analysis part 
of the study. It includes the development of a 
trade-off matrix to compare the alternatives, 
performance of a sensitivity analysis, selection 
of a preferred alternative, and a re-evaluation 
(sanity check) of the alternatives and the study 
process. Since weighting factors and some 
“quantified” data can have arbitrary aspects, the 
sensitivity analysis is crucial. If the solution can 
be changed with relatively minor changes in 
data input, the study is probably invalid, and 
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the methodology should be reviewed and 
revised. After the above tasks are complete, a 
solution is chosen, documented, and recorded 
in the database. 

Cost Effectiveness Analyses 

Cost effectiveness analyses are a special case trade 
study that compares system or component perfor¬ 
mance to its cost. These analyses help determine 
affordability and relative values of alternate 
solutions. Specifically, they are used to: 

• Support identification of affordable, cost opti¬ 
mized mission and performance requirements, 

• Support the allocation of performance to an 
optimum functional structure, 

• Provide criteria for the selection of alternative 
solutions, 


• Provide analytic confirmation that designs 
satisfy customer requirements within cost 
constraints, and 

• Support product and process verification. 

12.3 SUMMARY POINTS 

• The puipose of trade studies is to make better 
and more informed decisions in selecting best 
alternative solutions. 

• Initial trade studies focus on alternative system 
concepts and requirements. Later studies assist 
in selecting component part designs. 

• Cost effectiveness analyses provide assessments 
of alternative solution performance relative to 
cost. 
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SUPPLEMENT 12-A 

UTILITY CURVE 
METHODOLOGY 


The utility curve is a common methodology used 
in DoD and industry to perform trade-off analy¬ 
sis. In DoD it is widely used for cost effectiveness 
analysis and proposal evaluation. 

Utility Curve 

The method uses a utility curve, Figure 12-2, for 
each of the decision factors to normalize them to 
ease comparison. This method establishes the rela¬ 
tive value of the factor as it increases from the 
minimum value of the range. The curve shows can 
show a constant value relationship (straight line), 
increasing value (concave curve), decreasing value 
(convex curve), or a stepped value. 

Decision Matrix 

Each of the decision factors will also have relative 
value between them. These relative values are used 


to establish weighting factors for each decision 
factor. The weighting factors prioritize the deci¬ 
sion factors and allow direct comparison between 
them. A decision matrix, similar to Figure 12-3, is 
generated to evaluate the relative value of the 
alternative solutions. In the case of Figure 12-3 
range is given a weight of 2.0, speed a weight of 
1.0, and payload a weight of 2.5. The utility val¬ 
ues for each of the decision factors are multiplied 
by the appropriate weight. The weighted values 
for each alternative solution are added to obtain a 
total score for each solution. The solution with the 
highest score becomes the preferred solution. For 
the transport analysis of Figure 12-3 the apparent 
preferred solution is System 3. 

Sensitivity 

Figure 12-3 also illustrates a problem with the 
utility curve method. Both the utility curve and 


Utility 



Decision Factor 
(e.g., speed, cost, reliability, etc.) 


Figure 12-2. Utility Curve 
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weighting factors contain a degree of judgment that 
can vary between evaluators. Figure 12-3 shows 
three systems clustered around 3.8, indicating that 
a small variation in the utility curve or weighting 
factor could change the results. In the case of Fig¬ 
ure 12-3, a sensitivity analysis should be performed 
to determine how solutions change as utility and 
weighting change. This will guide the evaluator in 
determining how to adjust evaluation criteria to 
eliminate the problem’s sensitivity to small 
changes. In the case of Figure 12-3 the solution 
could be as simple as re-evaluating weighting fac¬ 
tors to express better the true value to the customer. 
For example, if the value of range is considered to 
be less and payload worth more than originally 
stated, then System 4 may become a clear winner. 


Notes 

When developing or adjusting utility curves and 
weighting factors, communication with the 
customers and decision makers is essential. Most 
sensitivity problems are not as obvious as Figure 
12-3. Sensitivity need not be apparent in the alter¬ 
natives’ total score. To ensure study viability, 
sensitivity analysis should always be done to 
examine the consequences of methodology choice. 
(Most decision support software provides a 
sensitivity analysis feature.) 


^^^IDecision Factors 

Range 

Speed 

Payload 



Wt. = 

= 2.0 

Wt. = 

: 1.0 

Wt. : 

= 2.5 

Weighted 

Total 

Alternatives 

U 

W 

U 

W 

U 

W 


Transport System 1 

.8 

1.6 

.7 

.7 

.6 

1.5 

3.8 

Transport System 2 

.7 

1.4 

.9 

.9 

.4 

1.0 

3.3 

Transport System 3 

.6 

1.2 

.7 

.7 

.8 

2.0 

3.9 

Transport System 4 

.5 

1.0 

.5 

.5 

.9 

2.25 

3.75 

Key: U = Utility value 

W = Weighted value 


Figure 12-3. Sample Decision Matrix 


116 























CHAPTER 13 


MODELING AND 
SIMULATION 


13.1 INTRODUCTION 

A model is a physical, mathematical, or logical 
representation of a system entity, phenomenon, or 
process. A simulation is the implementation of a 
model over time. A simulation brings a model to 
life and shows how a particular object or phenom¬ 
enon will behave. It is useful for testing, analysis 
or training where real-world systems or concepts 
can be represented by a model. 

Modeling and simulation (M&S) provides virtual 
duplication of products and processes, and 


represents those products or processes in readily 
available and operationally valid environments. 
Use of models and simulations can reduce the cost 
and risk of life cycle activities. As shown by Figure 
13-1, the advantages are significant throughout the 
life cycle. 

Modeling, Simulation, and Acquisition 

Modeling and simulation has become a very 
important tool across all acquisition-cycle phases 
and all applications: requirements definition; 
program management; design and engineering; 


$ Savings 


Prove System Need: 

Use existing high resolution 
models to emulate 
operational situation 


Smooth Transition to Operation 

• Manual proven 

• Trained personnel 

• Operationally ready before 
equipment is given to 
operators 


Saves Time 



Shortens 

Schedules 


Test “concepts” in the “real 
world” of simulation using 
simple models and putting 
operators into process 


Reduce Program Risks 

• Design 

• Integration 

• Transition to production 

• Testing 


Detail 

Design 




Prelim 

Design 


Improves IPPD 


Helps Refine Requirements 

• Get the user involved 

• Prevent gold-plating 


Sometimes it’s the only way 
to verify or validate 


Figure 13-1. Advantages of Modeling and Simulation 
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efficient test planning; result prediction; supple¬ 
ment to actual test and evaluation; manufacturing; 
and logistics support. With so many opportunities 
to use M&S, its four major benefits; cost savings, 
accelerated schedule, improved product quality and 
cost avoidance can be achieved in any system 
development when appropriately applied. DoD and 
industry around the world have recognized these 
opportunities, and many are taking advantage of 
the increasing capabilities of computer and infor¬ 
mation technology. M&S is now capable of 
prototyping full systems, networks, interconnect¬ 
ing multiple systems and their simulators so that 
simulation technology is moving in every direction 
conceivable. 


13.2 CLASSES OF SIMULATIONS 

The three classes of models and simulations are 
virtual, constructive, and live: 

• Virtual simulations represent systems both 
physically and electronically. Examples are air¬ 
craft trainers, the Navy’s Battle Force Tactical 
Trainer, Close Combat Tactical Trainer, and 
built-in training. 

• Constructive simulations represent a system 
and its employment. They include computer 
models, analytic tools, mockups, IDEF, Flow 
Diagrams, and Computer-Aided Design/ Manu¬ 
facturing (CAD/CAM). 

• Live simulations are simulated operations with 
real operators and real equipment. Examples 
are fire drills, operational tests, and initial 
production run with soft tooling. 

Virtual Simulation 

Virtual simulations put the human-in-the-loop. The 
operator’s physical interface with the system is 
duplicated, and the simulated system is made to 
perform as if it were the real system. The operator 
is subjected to an environment that looks, feels, 
and behaves like the real thing. The more advanced 
version of this is the virtual prototype, which allows 
the individual to interface with a virtual mockup 


operating in a realistic computer-generated envir¬ 
onment. A virtual prototype is a computer-based 
simulation of a system or subsystem with a degree 
of functional realism that is comparable to that of 
a physical prototype. 

Constructive Simulations 

The puipose of systems engineering is to develop 
descriptions of system solutions. Accordingly, con¬ 
structive simulations are important products in all 
key system engineering tasks and activities. Of 
special interest to the systems engineer are Com¬ 
puter-Aided Engineering (CAE) tools. Computer- 
aided tools can allow more in-depth and complete 
analysis of system requirements early in design. 
They can provide improved communication be¬ 
cause data can be disseminated rapidly to several 
individuals concurrently, and because design 
changes can be incorporated and distributed 
expeditiously. Key computer-aided engineering 
tools are CAD, CAE, CAM, Continuous Acquisi¬ 
tion and Life Cycle Support, and Computer-Aided 
Systems Engineering: 

Computer-Aided Design (CAD). CAD tools are 
used to describe the product electronically to 
facilitate and support design decisions. It can model 
diverse aspects of the system such as how compo¬ 
nents can be laid out on electrical/electronic cir¬ 
cuit boards, how piping or conduit is routed, or 
how diagnostics will be performed. It is used to 
lay out systems or components for sizing, posi¬ 
tioning, and space allocating using two- or three- 
dimensional displays. It uses three-dimensional 
“solid” models to ensure that assemblies, surfaces, 
intersections, interfaces, etc., are clearly defined. 
Most CAD tools automatically generate isometric 
and exploded views of detailed dimensional and 
assembly drawings, and determine component sur¬ 
face areas, volumes, weights, moments of inertia, 
centers of gravity, etc. Additionally, many CAD 
tools can develop three-dimensional models of 
facilities, operator consoles, maintenance work¬ 
stations, etc., for evaluating man-machine inter¬ 
faces. CAD tools are available in numerous vari¬ 
eties, reflecting different degrees of capabilities, 
fidelity, and cost. The commercial CAD/CAM 
product, Computer-Aided Three-Dimensional 
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Interactive Application (CATIA), was used to 
develop the Boeing 777, and is a good example of 
current state-of-the-art CAD. 

Computer-Aided Engineering (CAE). CAE pro¬ 
vides automation of requirements and performance 
analyses in support of trade studies. It normally 
would automate technical analyses such as stress, 
thermodynamic, acoustic, vibration, or heat trans¬ 
fer analysis. Additionally, it can provide automated 
processes for functional analyses such as fault 
isolation and testing, failure mode, and safety 
analyses. CAE can also provide automation of life- 
cycle-oriented analysis necessary to support the 
design. Maintainability, producibility, human fac¬ 
tor, logistics support, and value/cost analyses are 
available with CAE tools. 

Computer-Aided Manufacturing (CAM). CAM 
tools are generally designed to provide automated 
support to both production process planning and 
to the project management process. Process plan¬ 
ning attributes of CAM include establishing 
Numerical Control parameters, controlling 
machine tools using pre-coded instructions, pro¬ 
gramming robotic machinery, handling material, 
and ordering replacement parts. The production 
management aspect of CAM provides management 
control over production-relevant data, uses histori¬ 
cal actual costs to predict cost and plan activities, 
identifies schedule slips or slack on a daily basis, 
and tracks metrics relative to procurement, 
inventory, forecasting, scheduling, cost reporting, 
support, quality, maintenance, capacity, etc. A com¬ 
mon example of a computer-based project plan¬ 
ning and control tool is Manufacturing Resource 
Planning II (MRP II). Some CAM programs can 
accept data direct from a CAD program. With this 
type of tool, generally referred to as CAD/CAM, 
substantial CAM data is automatically generated 
by importing the CAD data directly into the CAM 
software. 

Computer-Aided Systems Engineering (CASE). 
CASE tools provide automated support for the 
Systems Engineering and associated processes. 
CASE tools can provide automated support for 
integrating system engineering activities, perform¬ 
ing the systems engineering tasks outlined in 


previous chapters, and performing the systems 
analysis and control activities. It provides techni¬ 
cal management support and has a broader 
capability than either CAD or CAE. An increas¬ 
ing variety of CASE tools are available, as 
competition brings more products to market, and 
many of these support the commercial “best 
Systems Engineering practices.” 

Continuous Acquisition and Life Cycle Support 
(CALS). CALS relates to the application of 
computerized technology to plan and implement 
support functions. The emphasis is on information 
relating to maintenance, supply support, and asso¬ 
ciated functions. An important aspect of CALS is 
the importation of information developed during 
design and production. A key CALS function is to 
support the maintenance of the system configura¬ 
tion during the operation and support phase. In 
DoD, CALS supports activities of the logistics 
community rather than the specific program office, 
and transfer of data between the CAD or CAM 
programs to CALS has been problematic. As a 
result there is current emphasis on development of 
standards for compatible data exchange. Formats 
of import include: two- and three-dimensional 
models (CAD), ASCII formats (Technical Manu¬ 
als), two-dimensional illustrations (Technical 
Manuals), and Engineering Drawing formats (Ras¬ 
ter, Aperture cards). These formats will be employ¬ 
ed in the Integrated Data Environment (IDE) that 
is mandated for use in DoD program offices. 

Live Simulation 

Live simulations are simulated operations of real 
systems using real people in realistic situations. 
The intent is to put the system, including its 
operators, through an operational scenario, where 
some conditions and environments are mimicked 
to provide a realistic operating situation. Examples 
of live simulations range from fleet exercises to 
fire drills. 

Eventually live simulations must be performed to 
validate constructive and virtual simulations. How¬ 
ever, live simulations are usually costly, and trade 
studies should be performed to support the bal¬ 
ance of simulation types chosen for the program. 
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13.3 HARDWARE VERSUS SOFTWARE 

Though current emphasis is on software M&S, the 
decision of whether to use hardware, software, or 
a combined approach is dependent on the com¬ 
plexity of the system, the flexibility needed for the 
simulation, the level of fidelity required, and the 
potential for reuse. Software capabilities are 
increasing, making software solutions cost effec¬ 
tive for large complex projects and repeated pro¬ 
cesses. Hardware methods are particularly useful 
for validation of software M&S, simple or one¬ 
time projects, and quick checks on changes of pro¬ 
duction systems. M&S methods will vary widely 
in cost. Analysis of the cost-versus-benefits of 
potential M&S methods should be performed to 
support planning decisions. 

13.4 VERIFICATION, VALIDATION, 

AND ACCREDITATION 

How can you trust the model or simulation? 
Establish confidence in your model or simulation 
through formal verification, validation, and 
accreditation (VV&A). VV&A is usually identified 
with software, but the basic concept applies to 


hardware as well. Figure 13-2 shows the basic 

differences between the terms (VV&A). 

More specifically: 

• Verification is the process of determining that 
a model implementation accurately represents 
the developer’s conceptual description and 
specifications that the model was designed to. 

• Validation is the process of determining the 
manner and degree to which a model is an ac¬ 
curate representation of the real world from the 
perspective of the intended uses of the model, 
and of establishing the level of confidence that 
should be placed on this assessment. 

• Accreditation is the formal certification that a 
model or simulation is acceptable for use for a 
specific purpose. Accreditation is conferred by 
the organization best positioned to make the 
judgment that the model or simulation in 
question is acceptable. That organization may 
be an operational user, the program office, or a 
contractor, depending upon the purposes 
intended. 


Verification 



Developer 

Verification Agent 


Validation 


Accreditation 



Functional Expert 
Validation Agent 


Requester/User 

Accreditation Agent 


As design matures, re-examine basic assumptions. 


Figure 13-2. Verification, Validation, and Accreditation 
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VV&A is particularly necessary in cases where: 

• Complex and critical interoperability is being 
represented, 

• Reuse is intended, 

• Safety of life is involved, and 

• Significant resources are involved. 

VV&A Currency 

VV&A is applied at initial development and use. 
The VV&A process is required for all DoD simu¬ 
lations and should be redone whenever existing 
models and simulations undergo a major upgrade 
or modification. Additionally, whenever the model 
or simulation violates its documented methodol¬ 
ogy or inherent boundaries that were used to vali¬ 
date or verify by its different use, then VV&A must 
be redone. Accreditation, however, may remain 
valid for the specific application unless revoked 
by the Accreditation Agent, as long as its use or 
what it simulates doesn’t change. 

13.5 CONSIDERATIONS 

There are a number of considerations that should 
enter into decisions regarding the acquisition and 
employment of modeling and simulation in defense 
acquisition management. Among these are such 
concerns as cost, fidelity, planning, balance, and 
integration. 

Cost Versus Fidelity 

Fidelity is the degree to which aspects of the real 
world are represented in M&S. It is the founda¬ 
tion for development of the model and subsequent 
VV&A. Cost effectiveness is a serious issue with 
simulation fidelity, because fidelity can be an 
aggressive cost driver. The con cct balance between 
cost and fidelity should be the result of simulation 
need analysis. M&S designers and VV&A agents 
must decide when enough is enough. Fidelity needs 
can vary throughout the simulation. This variance 
should be identified by analysis and planned for. 


Note of caution: Don’t confuse the quality of the 
display with the quality of meeting simulation 
needs! An example of fidelity is a well-known 
flight simulator using a PC and simple joystick 
versus a full 6-degree of freedom fully-instru¬ 
mented aircraft cockpit. Both have value at differ¬ 
ent stages of flight training, but obviously vary 
significantly in cost from thousands of dollars to 
millions. This cost difference is based on fidelity, 
or degree of real-world accuracy. 

Planning 

Planning should be an inherent part of M&S, and, 
therefore, it must be proactive, early, continuous, 
and regular. Early planning will help achieve bal¬ 
ance and beneficial reuse and integration. With 
computer and simulation technologies evolving so 
rapidly, planning is a dynamic process. It must be 
a continuing process, and it is important that the 
appropriate simulation experts be involved to maxi¬ 
mize the use of new capabilities. M&S activities 
should be a paid of the integrated teaming and in¬ 
volve all responsible organizations. Integrated 
teams must develop their M&S plans and insert 
them into the overall planning process, including 
the TEMP, acquisition strategy, and any other 
program planning activity. 

M&S planning should include: 

• Identification of activities responsible for each 
VV&A element of each model or simulation, 
and 

• Thorough VV & A estimates, formally agreed to 
by all activities involved in M&S, including 
T&E commitments from the developmental 
testers, operational testers, and separate VV&A 
agents. 

Those responsible for the VV&A activities must 
be identified as a normal part of planning. Figure 
13-2 shows the developer as the verification agent, 
the functional expert as the validation agent, and 
the user as the accreditation agent. In general this 
is appropriate for virtual simulations. However, the 
manufacturer of a constructive simulation would 
usually be expected to justify or warrantee their 


121 



Systems Engineering Fundamentals 


Chapter 13 


program’s use for a particular application. The 
question of who should actually accomplish 
VV&A is one that is answered in planning. VV&A 
requirements should be specifically called out in 
tasking documents and contracts. When appropri¬ 
ate, VV&A should be part of the contractor’s 
proposal, and negotiated prior to contract award. 

Balance 

Balance refers to the use of M&S across the phases 
of the product life cycle and across the spectrum 
of functional disciplines involved. The term may 
further refer to the use of hardware versus soft¬ 
ware, fidelity level, VV&A level, and even use 
versus non-use. Balance should always be based 
on cost effectiveness analysis. Cost effectiveness 
analyses should be comprehensive; that is, M&S 
should be properly considered for use in all paral¬ 
lel applications and across the complete life cycle 
of the system development and use. 


Integration 

Integration is obtained by designing a model or 
simulation to inter-operate with other models or 
simulations for the purpose of increased perfor¬ 
mance, cost benefit, or synergism. Multiple ben¬ 
efits or savings can be gained from increased 
synergism and use over time and across activities. 
Integration is achieved through reuse or upgrade 
of legacy programs used by the system, or of the 
proactive planning of integrated development of 
new simulations. In this case integration is accom¬ 
plished through the planned utilization of models, 
simulations, or data for multiple times or applica¬ 
tions over the system life cycle. The planned 
upgrade of M&S for evolving or parallel uses 
supports the application of open systems architec¬ 
ture to the system design. M&S efforts that are 
established to perform a specific function by a 
specific contractor, subcontractor, or government 
activity will tend to be sub-optimized. To achieve 
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integration M&S should be managed at least at the 
program office level. 

The Future Direction 

DoD, the Services, and their commands have 
strongly endorsed the use of M&S throughout the 
acquisition life cycle. The supporting simulation 
technology is also evolving as fast as computer 
technology changes, providing greater fidelity and 
flexibility. As more simulations are interconnected, 
the opportunities for further integration expand. 
M&S successes to date also accelerate its use. The 
current focus is to achieve open systems of simu¬ 
lations, so they can be plug-and-play across the 
spectrum of applications. From concept analysis 
through disposal analysis, programs may use hun¬ 
dreds of different simulations, simulators and 
model analysis tools. Figure 13-3 shows concep¬ 
tually how an integrated program M&S would 
affect the functions of the acquisition process. 

A formal DoD initiative, Simulation Based Acqui¬ 
sition (SBA), is currently underway. The SBA 
vision is to advance the implementation of M&S 
in the DoD acquisition process toward a robust, 
collaborative use of simulation technology that is 
integrated across acquisition phases and programs. 
The result will be programs that are much better 
integrated in an IPPD sense, and which are much 
more efficient in the use of time and dollars 
expended to meet the needs of operational users. 


13.6 SUMMARY 

• M&S provides virtual duplication of products 
and processes, and represent those products or 
processes in readily available and operationally 
valid environments. 

• M&S should be applied throughout the system 
life cycle in support of systems engineering 
activities. 

• The three classes of models and simulations are 
virtual, constructive, and live. 

• Establish confidence in your model or simula¬ 
tion through formal VV&A. 

• M&S planning should be an inherent part of 
Systems Engineering planning, and, therefore, 
pro-active, early, continuous, and regular. 

• A more detailed discussion of the use and man¬ 
agement of M&S in DoD acquisition is avail¬ 
able in the DSMC publication Systems Acqui¬ 
sition Manager’s Guide for the Use of Models 
and Simulations. 

• An excellent second source is the DSMC pub¬ 
lication, Simulation Based Acquisition -A New 
Approach. It surveys applications of increas¬ 
ing integration of simulation in current DoD 
programs and the resulting increasing benefits 
through greater integration. 
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14.1 METRICS IN MANAGEMENT 

Metrics are measurements collected for the pur¬ 
pose of determining project progress and overall 
condition by observing the change of the measured 
quantity over time. Management of technical 
activities requires use of three basic types of 
metrics: 

• Product metrics that track the development of 
the product, 

• Earned Value which tracks conformance to the 
planned schedule and cost, and 

• Management process metrics that track 
management activities. 

Measurement, evaluation and control of metrics is 
accomplished through a system of periodic report¬ 
ing must be planned, established, and monitored 
to assure metrics are properly measured, evaluated, 
and the resulting data disseminated. 

Product Metrics 

Product metrics are those that track key attributes 
of the design to observe progress toward meeting 
customer requirements. Product metrics reflect 
three basic types of requirements: operational per¬ 
formance, life-cycle suitability, and affordability. 
The key set of systems engineering metrics are the 
Technical Performance Measurements (TPM.) 
TPMs are product metrics that track design 
progress toward meeting customer performance 
requirements. They are closely associated with the 
system engineering process because they directly 
support traceability of operational needs to the 
design effort. TPMs are derived from Measures of 
Performance (MOPs) which reflect system require¬ 
ments. MOPs are derived from Measures of 


Effectiveness (MOEs) which reflect operational 
performance requirements. 

The term “metric” implies quantitatively measur¬ 
able data. In design, the usefulness of metric data 
is greater if it can be measured at the configura¬ 
tion item level. For example, weight can be esti¬ 
mated at all levels of the WBS. Speed, though an 
extremely important operational parameter, can¬ 
not be allocated down through the WBS. It cannot 
be measured, except through analysis and simula¬ 
tion, until an integrated product is available. Since 
weight is an important factor in achieving speed 
objectives, and weight can be measured at various 
levels as the system is being developed, weight 
may be the better choice as a metric. It has a direct 
impact on speed, so it traces to the operational 
requirement, but, most importantly, it can be allo¬ 
cated throughout the WBS and progress toward 
achieving weight goals may then be tracked 
through development to production. 

Measures of Effectiveness and Suitability 

Measures of Effectiveness (MOEs) and Measures 
of Suitability (MOSs) are measures of operational 
effectiveness and suitability in terms of operational 
outcomes. They identify the most critical perfor¬ 
mance requirements to meet system-level mission 
objectives, and will reflect key operational needs 
in the operational requirements document. 

Operational effectiveness is the overall degree of 
a system’s capability to achieve mission success 
considering the total operational environment. For 
example, weapon system effectiveness would con¬ 
sider environmental factors such as operator orga¬ 
nization, doctrine, and tactics; survivability; vul¬ 
nerability; and threat characteristics. MOSs, on 
the other hand, would measure the extent to which 
the system integrates well into the operation 


125 



Systems Engineering Fundamentals 


Chapter 14 


environment and would consider such issues as 
supportability, human interface compatibility, and 
maintainability. 

Measures of Performance 

MOPs characterize physical or functional attributes 
relating to the execution of the mission or func¬ 
tion. They quantify a technical or performance 
requirement directly derived from MOEs and 
MOSs. MOPs should relate to these measures such 
that a change in MOP can be related to a change in 
MOE or MOS. MOPs should also reflect key per¬ 
formance requirements in the system specification. 
MOPs are used to derive, develop, support, and 
document the performance requirements that will 
be the basis for design activities and process 
development. They also identify the critical tech¬ 
nical parameters that will be tracked through 
TPMs. 

Technical Performance Measurements 

TPMs are derived directly from MOPs, and are 
selected as being critical from a periodic review 
and control standpoint. TPMs help assess design 
progress, assess compliance to requirements 
throughout the WBS, and assist in monitoring and 
fracking technical risk. They can identify the need 
for deficiency recovery, and provide information 
to support cost-performance sensitivity assess¬ 
ments. TPMs can include range, accuracy, weight, 
size, availability, power output, power required, 
process time, and other product characteristics 
that relate directly to the system operational 
requirements. 

TPMs traceable to WBS elements are preferred, 
so elements within the system can be monitored 
as well as the system as a whole. However, some 
necessary TPMs will be limited to the system or 
subsystem level. For example, the specific fuel 
consumption of an engine would be a TPM neces¬ 
sary to track during the engine development, but it 
is not allocated throughout the WBS. It is reported 
as a single data item reflecting the performance of 
the engine as a whole. In this case the metric will 
indicate that the design approach is consistent with 


the required performance, but it may not be useful 
as an early warning device to indicate progress 
toward meeting the design goal. A more detailed 
discussion of TPMs is available as Supplement A 
to this chapter. 

Example of Measures 

MOE: The vehicle must be able to drive fully 
loaded from Washington, DC, to Tampa on one 
tank of fuel. 

MOP: Vehicle range must be equal to or greater 
than 1,000 miles. 

TPM: Fuel consumption, vehicle weight, tank size, 
drag, power train friction, etc. 

Suitability Metrics 

Tracking metrics relating to operational suitabil¬ 
ity and other life cycle concerns may be appropri¬ 
ate to monitor progress toward an integrated design. 
Operational suitability is the degree to which a 
system can be placed satisfactorily in field use 
considering availability, compatibility, transport¬ 
ability, interoperability, reliability, usage rates, 
maintainability, safety, human factors, documen¬ 
tation, training, manpower, supportability, logis¬ 
tics, and environmental impacts. These suitability 
parameters can generate product metrics that 
indicate progress toward an operationally suitable 
system. For example, factors that indicate the 
level of automation in the design would reflect 
progress toward achieving manpower quantity and 
quality requirements. TPMs and suitability prod¬ 
uct metrics commonly overlap. For example, Mean 
Time Between Failure (MBTF) can reflect both 
effectiveness or suitability requirements. 

Suitability metrics would also include measure¬ 
ments that indicate improvement in the produci- 
bility, testability, degree of design simplicity, and 
design robustness. For example, tracking number 
of parts, number of like parts, and number of weal¬ 
ing parts provides indicators of producibility, 
maintainability, and design simplicity. 
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Product Affordability Metrics 

Estimated unit production cost can be tracked 
during the design effort in a manner similar to the 
TPM approach, with each Cl element reporting an 
estimate based on current design. These estimates 
are combined at higher WBS levels to provide 
subsystem and system cost estimates. This provides 
a running engineering estimate of unit production 
cost, tracking of conformance to Design-to-Cost 
(DTC) goals, and a method to isolate design 
problems relating to production costs. 

Life cycle affordability can be tracked through 
factors that are significant in parametric life cycle 
cost calculations for the particular system. For 
example, two factors that reflect life cycle cost for 
most transport systems are fuel consumption and 
weight, both of which can be Packed as metrics. 

Timing 

Product metrics are tied directly to the design pro¬ 
cess. Planning for metric identification, reporting, 
and analysis is begun with initial planning in the 
concept exploration phase. The earliest systems 
engineering planning should define the manage¬ 
ment approach, identify performance or charac¬ 
teristics to be measured and Packed, forecast values 
for those performances or characteristics, deter¬ 
mine when assessments will be done, and establish 
the objectives of assessment. 

Implementation is begun with the development of 
the functional baseline. During this period, sys¬ 
tems engineering planning will identify critical 
technical parameters, time phase planned profiles 
with tolerance bands and thresholds, reviews or 
audits or events dependent or critical for achieve¬ 
ment of planned profiles, and the method of esti¬ 
mation. During the design effort, from functional 
to product baseline, the plan will be implemented 
and continually updated by the systems engineer¬ 
ing process. To support implementation, contracts 
should include provision for contractors to provide 
measurement, analysis, and reporting. The need 
to Pack product metrics ends in the production 
phase, usually concurrent with the establishment 
of the product (as built) baseline. 


DoD and Industry Policy on Product Metrics 

Analysis and control activities shall include 
performance metrics to measure technical 
development and design, actual versus planned; 
and to measure [the extent to which systems meet 
requirements]. DoD 5000.2-R. 

The performing activity establishes and imple¬ 
ments TPM to evaluate the adequacy of evolving 
solutions to identify deficiencies impacting the 
ability of the system to satisfy a designated value 
for a technical parameter. EIA IS-632, Section 3. 

The performing activity identifies the technical 
performance measures which are key indicators 
of system performance...should be limited to 
critical MOPs which, if not met put the project at 
cost, schedule, or performance risk. IEEE 1220, 
Section 6. 


14.2 EARNED VALUE 

Earned Value is a metric reporting system that uses 
cost-performance metrics to track the cost and 
schedule progress of system development against 
a projected baseline. It is a “big picture” approach 
and integrates concerns related to performance, 
cost, and schedule. Referring to Figure 14-1, if we 
think of the line labeled BCWP (budgeted cost of 
work performed) as the value that the conPactor 
has “earned,” then deviations from this baseline 
indicate problems in either cost or schedule. For 
example, if actual costs vary from budgeted costs, 
we have a cost variance; if work performed varies 
from work planned, we have a schedule variance. 
The projected performance is based on estimates 
of appropriate cost and schedule to perform the 
work required by each WBS element. When a vari¬ 
ance occurs the system engineer can pinpoint WBS 
elements that have potential technical development 
problems. Combined with product mePics, earned 
value is a powerful technical management tool 
for detecting and understanding development 
problems. 

Relationships exist between product metrics, the 
event schedule, the calendar schedule, and Earned 
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Time Completion 

Now Date 


Figure 14-1. Earned Value Concept 


Value: 

• The Event Schedule includes tasks for each 
event/exit criteria that must be performed to 
meet key system requirements, which are 
directly related to product metrics. 

• The Calendar (Detail) Schedule includes time 
frames established to meet those same product 
metric-related objectives (schedules). 

• Earned Value includes cost/schedule impacts 
of not meeting those objectives, and, when 
correlated with product metrics, can identify 
emerging program and technical risk. 


14.3 PROCESS METRICS 

Management process metrics are measurements 
taken to track the process of developing, building, 
and introducing the system. They include a wide 
range of potential factors and selection is pro¬ 
gram unique. They measure such factors as 
availability of resources, activity time rates, items 
completed, completion rates, and customer or team 
satisfaction. 


Examples of these factors are: number of trained 
personnel onboard, average time to approve/dis¬ 
approve ECPs, lines of code or drawings released, 
ECPs resolved per month, and team risk identifi¬ 
cation or feedback assessments. Selection of ap¬ 
propriate metrics should be done to track key man¬ 
agement activities. Selection of these metrics is 
part of the systems engineering planning process. 

How Much Metrics? 

The choice of the amount and depth of metrics is a 
planning function that seeks a balance between risk 
and cost. It depends on many considerations, in¬ 
cluding system complexity, organizational com¬ 
plexity, reporting frequency, how many contrac¬ 
tors, program office size and make up, contractor 
past performance, political visibility, and contract 
type. 

14.4 SUMMARY POINTS 

• Management of technical activities requires use 
of three basic types of metrics: product metrics 
that track the development of the product, 
earned value which tracks conformance to the 
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planned schedule and cost, and management 
process metrics that track management activi¬ 
ties. 

• Measurement, evaluation and control of metrics 
is accomplished through a system of periodic 
reporting that must be planned, established, and 
monitored to assure metrics are measured 
properly, evaluated, and the resulting data 
disseminated. 


• TPMs are performance based product metrics 
that track progress through measurement of key 
technical parameters. They are important to the 
systems engineering process because they con¬ 
nect operational requirements to measurable 
design characteristics and help assess how well 
the effort is meeting those requirements. TPMs 
are required for all programs covered by DoD 
5000.2-R. 
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SUPPLEMENT 14-A 

TECHNICAL PERFORMANCE 
MEASUREMENT 


Technical Performance Measurement (TPM) is an 
analysis and control technique that is used to: (1) 
project the probable performance of a selected 
technical parameter over a period of time, (2) 
record the actual performance observed of the 
selected parameter, and (3) through comparison 
of actual versus projected performance, assist the 
manager in decision making. A well thought out 
program of technical performance measures pro¬ 
vides an early warning of technical problems and 
supports assessments of the extent to which 
operational requirements will be met, as well as 
assessments of the impacts of proposed changes 
in system performance. 


TPMs generally take the form of both graphic dis¬ 
plays and narrative explanations. The graphic, an 
example of which is shown in Figure 14-2, shows 
the projected behavior of the selected parameter 
as a function of time, and further shows actual ob¬ 
servations, so that deviations from the planned pro¬ 
file can be assessed. The narrative portion of the 
report should explain the graphic, addressing the 
reasons for deviations from the planned profile, 
assessing the seriousness of those deviations, ex¬ 
plaining actions underway to correct the situation 
if required, and projecting future performance, 
given the current situation. 


Technical 

Parameter 

Value 

e.g., Weight 


Planned 



Figure 14-2.Technical Performance Measurement-The Concept 
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Parameters to be tracked are typically based on 
the combined needs of the government and the 
contractor. The government program office will 
need a set of TPMs which provide visibility into 
the technical performance of key elements of the 
WBS, especially those which are cost drivers on 
the program, lie on the critical path, or which 
represent high risk items. 

The TPMs selected for delivery to the government 
are expected to be traceable to the needs of the 
operational user. The contractor will generally track 
more items than are reported to the government, 
as the contractor needs information at a more 
detailed level than does the government program 
office. 

TPM reporting to the government is a contractual 
issue, and those TPMs on which the government 
receives reports are defined as contract deliverables 
in the contract data requirements list. Which para¬ 
meters are selected for reporting depends on a num¬ 
ber of issues, among which are resources to pur¬ 
chase TPMs, the availability of people to review 
and follow the items, the complexity of the sys¬ 
tem involved, the phase of development, and the 
contractor’s past experience with similar systems. 

A typical TPM graphic will take a form somewhat 
like that previously shown. The actual form of the 
projected performance profile and whether or not 
tolerance bands are employed will be a function 
of the parameter selected and the needs of the pro¬ 
gram office. 

Another important consideration is the relation¬ 
ship between the TPM program and risk manage¬ 
ment. Generally, the parameters selected for track¬ 
ing should be related to the risk areas on the pro¬ 
gram. If a particular element of the design has been 
identified as a risk area, then parameters should 
be selected which will enable the manager to track 
progress in that area. For example, if achieving a 
required aircraft range is considered to be critical 
and a risk area, then tracking parameters that pro¬ 
vide insight into range would be selected, such as 
aircraft weight, specific fuel consumption, drag, 
etc. Furthermore, there should be consistency be¬ 
tween TPMs and the Critical Technical Parameters 


associated with formal testing, although the TPM 
program will not normally be limited just to those 
parameters identified as critical for test purposes. 

Government review and follow up of TPMs are 
appropriate on a periodic basis when submitted by 
the contractor, and at other major technical events 
such as at technical reviews, test events, and 
program management reviews. 

While TPMs are expected to be traceable to the 
needs of the user, they must be concrete technical 
parameters that can be projected and tracked. For 
example, an operational user may have a require¬ 
ment for survivability under combat conditions. 
Survivability is not, in and of itself, a measurable 
parameter, but there are important technical para¬ 
meters that determine survivability, such as radar 
cross section (RCS) and speed. Therefore, the tech¬ 
nical manager might select and track RCS and 
speed as elements for TPM reporting. The deci¬ 
sion on selection of parameters for TPM tracking 
must also take into consideration the extent to 
which the parameter behavior can be projected 
(profiled over a time period) and whether or not it 
can actually be measured. If the parameter cannot 
be profiled, measured, or is not critical to program 
success, then the government, in general, should 
not select it for TPM tracking. The WBS structure 
makes an excellent starting point for consideration 
of parameters for TPM tracking (see Figure 14-3). 

A substantial effort has taken place in recent years 
to link TPMs with Earned Value Management in a 
way that would result in earned value calculations 
that reflect the risks associated with achieving tech¬ 
nical performance. The approach used establishes 
statistical probability of achieving a projected level 
of performance on the TPM profile based on a 
statistical analysis of actual versus planned per¬ 
formance. Further information is available on the 
Internet at http://www.acq.osd.mil/api/tpm/ . 

In summary, TPMs are an important tool in the 
program manager’s systems analysis and control 
toolkit. They provide an early warning about de¬ 
viations in key technical parameters, which, if not 
controlled, can impact system success in meeting 
user needs. TPMs should be an integral paid of both 
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Figure 14-3. Shipboard Fire Control System (Partial) 


periodic program reporting and management fol- manager, whether technically grounded or not, can 
low-up, as well as elements for discussion in tech- make perceptive judgments about system techni- 
nical reviews and program management reviews. cal performance and can follow up on contractor 
By thoughtful use of a good program of TPM, the plans and progress when deviations occur. 


Relevant Terms 

Achievement to date - Measured or estimated progress plotted and compared with planned 

progress by designated milestone date. 

Current estimate - Expected value of a technical parameter at contract completion. 
Planned value - Predicted value of parameter at a given point in time. 

Planned profile - Time phased projected planned values. 

Tolerance band - Management alert limits representing projected level of estimating error. 

Threshold - Limiting acceptable value, usually contractual. 

Variance - Difference between the planned value and the achievement-to-date 
derived from analysis, test, or demonstration. 














CHAPTER 15 


RISK MANAGEMENT 


15.1 RISK AS REALITY 

Risk is inherent in all activities. It is a normal con¬ 
dition of existence. Risk is the potential for a nega¬ 
tive future reality that may or may not happen. Risk 
is defined by two characteristics of a possible nega¬ 
tive future event: probability of occurrence 
(whether something will happen), and conse¬ 
quences of occurrence (how catastrophic if it hap¬ 
pens). If the probability of occurrence is not known 
then one has uncertainty, and the risk is undefined. 

Risk is not a problem. It is an understanding of the 
level of threat due to potential problems. A prob¬ 
lem is a consequence that has already occurred. 

In fact, knowledge of a risk is an opportunity to 
avoid a problem. Risk occurs whether there is an 
attempt to manage it or not. Risk exists whether 
you acknowledge it, whether you believe it, 


whether if it is written down, or whether you 
understand it. Risk does not change because you 
hope it will, you ignore it, or your boss’s expecta¬ 
tions do not reflect it. Nor will it change just 
because it is contrary to policy, procedure, or 
regulation. Risk is neither good nor bad. It is just 
how things are. Progress and opportunity are 
companions of risk. In order to make progress, risks 
must be understood, managed, and reduced to 
acceptable levels. 

Types of Risk in a 

Systems Engineering Environment 

Systems engineering management related risks 
could be related to the system products or to the 
process of developing the system. Figure 15-1 
shows the decomposition of system development 
risks. 



Figure 15-1. Risk Hierarchy 
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Risks related to the system development generally 
are traceable to achieving life cycle customer 
requirements. Product risks include both end prod¬ 
uct risks that relate to the basic performance and 
cost of the system, and to enabling products that 
relate to the products that produce, maintain, 
support, test, train, and dispose of the system. 

Risks relating to the management of the develop¬ 
ment effort can be technical management risk or 
risk caused by external influences. Risks dealing 
with the internal technical management include 
those associated with schedules, resources, work 
flow, on time deliverables, availability of appro¬ 
priate personnel, potential bottlenecks, critical path 
operations and the like. Risks dealing with exter¬ 
nal influences include resource availability, higher 
authority delegation, level of program visibility, 
regulatory requirements, and the like. 

15.2 RISK MANAGEMENT 

Risk management is an organized method for iden¬ 
tifying and measuring risk and for selecting, 
developing, and implementing options for the 


handling of risk. It is a process, not a series of 
events. Risk management depends on risk man¬ 
agement planning, early identification and analy¬ 
sis of risks, continuous risk tracking and reassess¬ 
ment, early implementation of corrective actions, 
communication, documentation, and coordination. 
Though there are many ways to structure risk man¬ 
agement, this book will structure it as having four 
parts: Planning, Assessment, Handling, and Moni¬ 
toring. As depicted in Figure 15-2 all of the parts 
are interlocked to demonstrate that after initial 
planning the parts begin to be dependent on each 
other. Illustrating this, Figure 15-3 shows the key 
control and feedback relationships in the process. 

Risk Planning 

Risk Planning is the continuing process of devel¬ 
oping an organized, comprehensive approach to 
risk management. The initial planning includes 
establishing a strategy; establishing goals and 
objectives; planning assessment, handling, and 
monitoring activities; identifying resources, tasks, 
and responsibilities; organizing and training risk 
management IPT members; establishing a method 
to track risk items; and establishing a method to 
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Management 


Figure 15-3. Risk Management Control and Feedback 


document and disseminate information on a 
continuous basis. 

In a systems engineering environment risk plan¬ 
ning should be: 

• Inherent (imbedded) in systems engineering 
planning and other related planning, such as 
producibility, supportability, and configuration 
management; 

• A documented, continuous effort; 

• Integrated among all activities; 

• Integrated with other planning, such as systems 
engineering planning, supportability analysis, 
production planning, configuration and data 
management, etc.; 

• Integrated with previous and future phases; and 

• Selective for each Configuration Baseline. 

Risk is altered by time. As we try to control or 
alter risk, its probability and/or consequence will 


change. Judgment of the risk impact and the 
method of handling the risk must be reassessed 
and potentially altered as events unfold. Since these 
events are continually changing, the planning 
process is a continuous one. 

Risk Assessment 

Risk assessment consists of identifying and ana¬ 
lyzing the risks associated with the life cycle of 
the system. 

Risk Identification Activities 

Risk identification activities establish what risks 
are of concern. These activities include: 

• Identifying risk/uncertainty sources and drivers, 

• Transforming uncertainty into risk, 

• Quantifying risk, 

• Establishing probability, and 

• Establishing the priority of risk items. 
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As shown by Figure 15-4 the initial identification 
process starts with an identification of potential 
risk items in each of the four risk areas. Risks re¬ 
lated to the system performance and supporting 
products are generally organized by WBS and ini¬ 
tially determined by expert assessment of teams 
and individuals in the development enteiprise. 
These risks tend to be those that require follow-up 
quantitative assessment. Internal process and ex¬ 
ternal influence risks are also determined by ex¬ 
pert assessment within the enteiprise, as well as 
through the use of risk area templates similar to 
those found in DoD 4245.7-M. The DoD 4245.7- 
M templates describe the risk areas associated with 
system acquisition management processes, and 
provide methods for reducing traditional risks in 
each area. These templates should be tailored for 
specific program use based on expert feedback. 

After identifying the risk items, the risk level 
should be established. One common method is 
through the use of a matrix such as shown in Fig¬ 
ure 15-5. Each item is associated with a block in 
the matrix to establish relative risk among them. 


On such a graph risk increases on the diagonal and 
provides a method for assessing relative risk. Once 
the relative risk is known, a priority list can be 
established and risk analysis can begin. 

Risk identification efforts can also include activi¬ 
ties that help define the probability or consequences 
of a risk item, such as: 

• Testing and analyzing uncertainty away, 

• Testing to understand probability and conse¬ 
quences, and 

• Activities that quantify risk where the qualita¬ 
tive nature of high, moderate, low estimates are 
insufficient for adequate understanding. 

Risk Analysis Activities 

Risk analysis activities continue the assessment 
process by refining the description of identified 
risk event through isolation of the cause of risk, 
determination of the full impact of risk, and the 



Figure 15-4. Initial Risk Identificaiton 
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Figure 15-5. Simple Risk Matrix 


determination and choose of alternative courses of 
action. They are used to determine what risk should 
be tracked, what data is used to track risk, and what 
methods are used to handle the risk. 

Risk analysis explores the options, opportunities, 
and alternatives associated with the risk. It ad¬ 
dresses the questions of how many legitimate ways 
the risk could be dealt with and the best way to do 
so. It examines sensitivity, and risk interrelation¬ 
ships by analyzing impacts and sensitivity of 
related risks and performance variation. It further 
analyzes the impact of potential and accomplished, 
external and internal changes. 

Risk analysis activities that help define the scope 
and sensitivity of the risk item include finding 
answers to the following questions: 

• If something changes, will risk change faster, 
slower, or at the same pace? 

• If a given risk item occurs, what collateral 
effects happen? 

• How does it affect other risks? 


• How does it affect the overall situation? 

• Development of a watch list (prioritized list of 
risk items that demand constant attention by 
management) and a set of metrics to determine 
if risks are steady, increasing, or decreasing. 

• Development of a feedback system to track 
metrics and other risk management data. 

• Development of quantified risk assessment. 

Quantified risk assessment is a formal quantifica¬ 
tion of probabilities of occurrence and conse¬ 
quences using a top-down structured process 
following the WBS. For each element, risks are 
assessed through analysis, simulation and test to 
determine statistical probability and specific 
conditions caused by the occurrence of the 
consequence. 

Cautions in Risk Assessments 

Reliance solely on numerical values from simula¬ 
tions and analysis should be avoided. Do not lose 
sight of the actual source and consequences of the 
risks. Testing does not eliminate risk. It only 
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provides data to assess and analyze risk. Most of 
all. beware of manipulating relative numbers, such 
as ‘risk index” or “risk scales,” even when based 
on expert opinion, as quantified data. They are 
important information, but they are largely sub¬ 
jective and relative; they do not necessarily define 
risk accurately. Numbers such as these should 
always be the subject of a sensitivity analysis. 

Risk Handling 

Once the risks have been categorized and analyzed, 
the process of handling those risks is initiated. The 
prime puipose of risk handling activities is to miti¬ 
gate risk. Methods for doing this are numerous, 
but all fall into four basic categories: 

• Risk Avoidance, 

• Risk Control, 

• Risk Assumption, and 

• Risk Transfer. 

Avoidance 

To avoid risk, remove requirements that represent 
uncertainty and high risk (probability or conse¬ 
quence.) Avoidance includes trading off risk for 
performance or other capability, and it is a key 
activity during requirements analysis. Avoidance 
requires understanding of priorities in requirements 
and constraints. Are they mission critical, mission 
enhancing, nice to have, or “bells and whistles?” 

Control 

Control is the deliberate use of the design process 
to lower the risk to acceptable levels. It requires 
the disciplined application of the systems engi¬ 
neering process and detailed knowledge of the 
technical area associated with the design. Control 
techniques are plentiful and include: 

• Multiple concurrent design to provide more 
than one design path to a solution, 

• Alternative low-risk design to minimize the risk 
of a design solution by using the lowest-risk 
design option. 


• Incremental development, such as preplanned 
product improvement, to dissociate the design 
from high-risk components that can be devel¬ 
oped separately, 

• Technology maturation that allows high-risk 
components to be developed separately while 
the basic development uses a less risky and 
lower-performance temporary substitute, 

• Test, analyze and fix that allows understanding 
to lead to lower risk design changes. (Test can 
be replaced by demonstration, inspection, early 
prototyping, reviews, metric tracking, experi¬ 
mentation, models and mock-ups, simulation, 
or any other input or set of inputs that gives a 
better understanding of the risk), 

• Robust design that produces a design with sub¬ 
stantial margin such that risk is reduced, and 

• The open system approach that emphasizes use 
of generally accepted interface standards that 
provide proven solutions to component design 
problems. 

Acceptance 

Acceptance is the deliberate acceptance of the risk 
because it is low enough in probability and/or con¬ 
sequence to be reasonably assumed without 
impacting the development effort. Key techniques 
for handling accepted risk are budget and sched¬ 
ule reserves for unplanned activities and continu¬ 
ous assessment (to assure accepted risks are main¬ 
tained at acceptance level). The basic objective of 
risk management in systems engineering is to 
reduce ah risk to an acceptable level. 

The strong budgetary strain and tight schedules 
on DoD programs tends to reduce the program 
manager’s and system engineer’s capability to pro¬ 
vide reserve. By identifying a risk as acceptable, 
the worst-case outcome is being declared accept¬ 
able. Accordingly, the level of risk considered 
acceptable should be chosen very carefully in a 
DoD acquisition program. 
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Transfer 

Transfer can be used to reduce risk by moving the 
risk from one area of design to another where a 
design solution is less risky. Examples of this in¬ 
clude: 

• Assignment to hardware (versus software) or 
vice versa; and 

• Use of functional partitioning to allocate per¬ 
formance based on risk factors. 

Transfer is most associated with the act of assign¬ 
ing, delegating, or paying someone to assume the 
risk. To some extent transfer always occurs when 
contracting or tasking another activity. The con¬ 
tract or tasking document sets up agreements that 
can transfer risk from the government to contrac¬ 
tor, program office to agency, and vice versa. Typi¬ 
cal methods include insurance, warranties, and 
incentive clauses. Risk is never truly transferred. 
If the risk isn’t mitigated by the delegated activity 
it still affects your project or program. 

Key areas to review before using transfer are: 

• How well can the delegated activity handle the 
risk? Transfer is effective only to the level the 
risk taker can handle it. 

• How well will the delegated activity solution 
integrate into your project or program? Trans¬ 
fer is effective only if the method is integrated 
with the overall effort. For example, is the war¬ 
ranty action coordinated with operators and 
maintainers? 

• Was the method of tasking the delegated activ¬ 
ity proper? Transfer is effective only if the trans¬ 
fer mechanism is valid. For example, can in¬ 
centives be “gamed?” 

• Who has the most control over the risk? If the 
project or program has no or little control over 
the risk item, then transfer should be consid¬ 
ered to delegate the risk to those most likely to 
be able to control it. 


Monitoring and Reporting 

Risk monitoring is the continuous process of track¬ 
ing and evaluating the risk management process 
by metric reporting, enterprise feedback on watch 
list items, and regular enterprise input on poten¬ 
tial developing risks. (The metrics, watch lists, and 
feedback system are developed and maintained as 
an assessment activity.) The output of this process 
is then distributed throughout the enterprise, so that 
all those involved with the program are aware of 
the risks that affect their efforts and the system 
development as a whole. 

Special Case - Integration as Risk 

Integration of technologies in a complex system is 
a technology in itself! Technology integration dur¬ 
ing design may be a high-risk item. It is not nor¬ 
mally assessed or analyzed as a separately identi¬ 
fied risk item. If integration risks are not properly 
identified during development of the functional 
baseline, they will demonstrate themselves as 
serious problems in the development of the product 
baseline. 

Special Case - Software Risk 

Based on past history, software development is 
often a high-risk area. Among the causes of per¬ 
formance, schedule, and cost deficiencies have 
been: 

• Imperfect understanding of operational 
requirements and its translation into source 
instructions, 

• Risk tracking and handling, 

• Insufficient comprehension of interface 
constraints, and 

• Fack of sufficient qualified personnel. 

Risk Awareness 

All members of the enterprise developing the 
system must understand the need to pay atten¬ 
tion to the existence and changing nature of risk. 
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Consequences that are unanticipated can seriously 
disrupt a development effort. The uneasy feeling 
that something is wrong, despite assurances that 
all is fine may be valid. These kinds of intuitions 
have allowed humanity to survive the slings and 
arrows of outrageous fortune throughout history. 
Though generally viewed as non-analytical, these 
apprehensions should not be ignored. Experience 
indicates those non-specific warnings have validity, 
and should be quantified as soon as possible. 

15.3 SUMMARY POINTS 

• Risk is inherent in all activities. 

• Risk is composed of knowledge of two charac¬ 
teristics of a possible negative future event: 
probability of occurrence and consequences of 
occurrence. 


• Risk management is associated with a clear 
understanding of probability. 

• Risk management is an essential and integral 
part of technical program management (systems 
engineering). 

• Risks and uncertainties must be identified, 
analyzed, handled, and tracked. 

• There are four basic ways of handling risk: 
avoidance, transfer, acceptance, and control. 

• Program risks are classified as low, moderate, 
or high depending on consequences and 
probability of occurrence. Risk classification 
should be based on quantified data to the extent 
possible. 
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SUPPLEMENT 15-A 

RISK MANAGEMENT 
IN DOD ACQUISITION 


Policy 

DoD policy is quite clear in regard to risk 
management: it must be done. 

The PM shall identify the risk areas in the pro¬ 
gram and integrate risk management within overall 
program management. (DoD 5000.2-R.) 

In addition, DoDD 5000.4 identifies risk and cost 
analysis as a responsibility of the program manager. 

Risk Management View 

A DSMC study indicates that major programs 
which declared moderate risk at Milestone B have 
been more successful in terms of meeting cost and 
schedule goals than those which declared low risk 
(DSMC TR 2-95). This strongly implies that pro¬ 
gram offices that understand and respect risk man¬ 
agement will be more successful. For this reason, 
the program office needs to adopt a systems-level 
view of risk. The systems engineer provides this 
view. Systems Engineering is the cornerstone of 
program office risk management program because 
it is the connection to realistic assessment of prod¬ 
uct maturity and development, and the product is, 
in the final analysis, what system acquisition is 
really about. 

Flowever, the program office has external risks to 
deal with as well as the internal risks prevalent in 
the development process. The Systems Engineer 
has to provide the program manager internal risk 
data in a manner that aids the handling of the 
external risks. In short, the systems engineer must 
present bad news such that it is reasonable and 
compelling to higher levels of authority. See 
Chapter 20 for further discussion on this topic. 


Factoring Risk Management into the Process 

Risk management, as an integral part of the over¬ 
all program planning and management process, is 
enhanced by applying a controlled, consistent, 
approach to systems engineering and using inte¬ 
grated teams for both product development and 
management control. Programs should be transi¬ 
tioned to the next phase only if risk is at the appro¬ 
priate level. Know the risk drivers behind the esti¬ 
mates. By its nature there are always subjective 
aspects to assessing and analyzing risk at the sys¬ 
tem level, even though they tend to be represented 
as quantitative and/or analytically objective. 

Risk and Phases 

Risk management begins in the Concept and Tech¬ 
nology Development phase. During Concept Ex¬ 
ploration initial system level risk assessments are 
made. Unknown-unknowns, uncertainty, and some 
high-risk elements are normal and expected. When 
substantial technical risk exists, the Component 
Advanced Development stage is appropriate, and 
is included in the life-cycle process specifically as 
an opportunity to address and reduce risks to a level 
that are consistent with movement into systems 
acquisition. 

The S&T community has a number of vehicles 
available that are appropriate for examining tech¬ 
nology in application and for undertaking risk 
reduction activities. These include Advanced 
Technology Demonstrations, Advanced Concept 
Technology Demonstrations, as well as Joint 
Warfighting Experiments. The focus of the activi¬ 
ties undertaken during these risk reduction stages 
include: 
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• Testing, analyzing, or mitigating system and 
subsystem uncertainty and high risk out of the 
program. 

• Demonstrating technology sufficient to uncover 
system and subsystem unknown-unknowns 
(especially for integration). 

• Planning for risk management during the 
transition to and continuation of systems ac¬ 
quisition during the System Development and 
Demonstration phase, especially handling and 
tracking of moderate risk. 

System Development and Demonstration requires 
the application of product and manufacturing 
engineering, which can be disrupted if the tech¬ 
nology development is not sufficient to support 
engineering development. Risk management in 
during this phase emphasizes: 

• Reduction and control of moderate risks, 

• All risks under management including emerging 
ones, and 

• Maintenance of risk levels and reaction to 
problems. 

Objective Assessment of Technology 

The revised acquisition process has been deliber¬ 
ately structured to encourage and allow programs 
to progress through appropriate risk reduction 
stages and phases, based on an objective assess¬ 
ment of the maturity levels associated with the 
products and systems under development. It is 
therefore, particularly important that program 
managers and their staffs ensure that the decisions 
made regarding recommendations to proceed, and 
the paths to be taken, be based on as impartial and 
objective opinions as possible. The temptation is 
always to move ahead and not to delay to improve 
the robustness of a given product or system. When 
systems are hurried into engineering development 
and production, in spite of the fact that the under¬ 
lying technologies require further development. 


history indicates that the results will eventually 
show the fallacy of speed over common sense. And 
to fix the problem in later stages of development— 
or even after deployment—can be hugely expen¬ 
sive in terms of both monetary cost and human 
lives. 

The prevailing presumption at Milestone B is that 
the system is ready for engineering development. 
After this, the acquisition community generally 
assumes that risk is moderate to low, that the tech¬ 
nology is “available.” There is evidence to support 
the assertion that programs often progress into 
engineering development with risks that actually 
require substantial exploratory and applied re¬ 
search and development to bring them to the mod¬ 
erate levels of risk or lower. One approach that has 
proven successful in making objective risk assess¬ 
ments is the use of independent evaluation teams. 
Groups that have no pre-determined interest to 
protect or axe to grind are often capable of provid¬ 
ing excellent advice regarding the extent to which 
a system is ready to proceed to the next level of 
development and subsequent phases. 

Risk Classification on the 
System (Program) Level 

Classification definitions should be established 
early and remain consistent throughout the pro¬ 
gram. The program office should assess the risks 
of achieving performance, schedule, and cost in 
clear and accurate terms of both probability and 
consequence. Where there is disagreement about 
the risk, assessment efforts should be immediately 
increased. Confusion over risk is the worst pro¬ 
gram risk, because it puts in doubt the validity of 
the risk management process, and therefore, 
whether program reality is truly understood. 

The system level risk assessment requires integra¬ 
tion and interpretation of the quantified risk 
assessment of the parts. This requires reasonable 
judgement. Because integration increases the po¬ 
tential for risk, it is reasonable to assume overall 
risk is not better than the sum of objective data for 
the parts. 
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Reality Versus Expectations 

Program managers are burdened with the expecta¬ 
tions of superiors and others that have control over 
the program office’s environment. Pressure to ac¬ 
commodate these expectations is high. If the sys¬ 
tems engineer cannot communicate the reality of 
risk in terms that are understandable, acceptable, 
or sufficiently verifiable to management, then these 
pressures may override vertical communication of 
actual risk. 


Formal systems engineering with risk management 
incorporated can provide the verifiable informa¬ 
tion. Flowever, the systems engineer also has the 
responsibility to adequately explain probability and 
consequences such that the program manager can 
accept the reality of the risk and override higher 
level expectations. 

Uncertainty is a special case, and very dangerous 
in an atmosphere of high level expectations. Pre¬ 
sentation of uncertainty issues should strongly em¬ 
phasize consequences, show probability trends, and 
develop “most likely” alternatives for probability. 
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SUPPLEMENT 15-B 


MODEL FOR 
SYSTEM LEVEL 
RISK ASSESSMENT 


The following may be used to assist in making preliminary judgments regarding risk classifications: 



Low Risk 

Moderate Risk 

High Risk 

Consequences 

Insignificant cost, 
schedule, or technical 
impact 

Affects program 
objectives, cost, or 
schedule; however 
cost, schedule, 
performance are 
achievable 

Significant impact, 
requiring reserve or 
alternate courses of 

action to recover 

Probability of 
Occurrence 

Little or no estimated 

likelihood 

Probability sufficiently 
high to be of concern 
to management 

High likelihood of 

occurrence 

Extent of 

Demonstration 

Full-scale, integrated 
technology has been 
demonstrated 
previously 

Has been demonstrated 
but design changes, 
tests in relevant 
environments required 

Significant design 
changes required in 
order to achieve 
required/desired 
results 

Existence of 
Capability 

Capability exists in 
known products; 
requires integration 
into new system 

Capability exists, but 
not at performance 
levels required for 
new system 

Capability does not 
currently exist 


Also see Technology Readiness Levels matrix in Chapter 2 
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SYSTEMS ENGINEERING 
PLANNING 


16.1 WHY ENGINEERING PLANS? 

Systems engineering planning is an activity that 
has direct impact on acquisition planning decisions 
and establishes the feasible methods to achieve the 
acquisition objectives. Management uses it to: 

• Assure that all technical activities are identified 
and managed, 

• Communicate the technical approach to the 
broad development team, 

• Document decisions and technical implemen¬ 
tation, and 

• Establish the criteria to judge how well the 
system development effort is meeting customer 
and management needs. 

Systems engineering planning addresses the scope 
of the technical effort required to develop the sys¬ 
tem. The basic questions of “who will do what” 
and “when” are addressed. As a minimum, a tech¬ 
nical plan describes what must be accomplished, 
how systems engineering will be done, how the 
effort will be scheduled, what resources are needed, 
and how the systems engineering effort will be 
monitored and controlled. The planning effort 
results in a management-oriented document 
covering the implementation of program require¬ 
ments for system engineering, including technical 
management approaches for subsequent phases of 
the life cycle. In DoD it is an exercise done on a 
systems level by the government, and on a more 
detailed level by contractors. 


Technical/Systems Engineering Planning 

Technical planning may be documented in a sepa¬ 
rate engineering management plan or incorporated 
into a broad, integrated program management plan. 
This plan is first drafted at project or program 
inception during the early requirements analysis 
effort. Requirements analysis and technical plan¬ 
ning are inherently linked, because requirements 
analysis establishes an understanding of what must 
be provided. This understanding is fundamental 
to the development of detailed plans. 

To be of utility, systems engineering plans must 
be regularly updated. To support management de¬ 
cision making, major updates will usually occur 
at least just before major management milestone 
decisions. However, updates must be performed 
as necessary between management milestones to 
keep the plan sufficiently current to achieve its 
purpose of information, communication, and 
documentation. 


16.2 ELEMENTS OF TECHNICAL PLANS 

Technical plans should include sufficient informa¬ 
tion to document the puipose and method of the 
systems engineering effort. Plans should include 
the following: 

• An introduction that states the puipose of the 
engineering effort and a description of the 
system being developed, 

• A technical strategy description that ties the 
engineering effort to the higher-level manage¬ 
ment planning. 
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• A description of how the systems engineering 
process will be tailored and structured to 
complete the objectives stated in the strategy, 

• An organization plan that describes the 
organizational structure that will achieve the 
engineering objectives, and 

• A resource plan that identifies the estimated 
funding and schedule necessary to achieve the 
strategy. 

Introduction 

The introduction should include: 

Scope: The scope of the plan should provide 
information concerning what paid of the big pic¬ 
ture the plan covers. For example, if the plan were 
a DoD program office plan, it would emphasize 
control of the higher-level requirements, the system 
definition (functional baseline), and all activities 
necessary for system development. On the other 
hand, a contractor’s plan would emphasize control 
of lower-level requirements, preliminary and detail 
designs (allocated and product baselines), and 
activities required and limited by the contractual 
agreement. 

Description: The description of the system should: 

• Be limited to an executive summary describing 
those features that make the system unique, 

• Include a general discussion of the system’s 
operational functions, and 

• Answer the question “What is it and what will 
it do?” 

Focus: A guiding focus for the effort should be 
provided to clarify the management vision for the 
development approach. For example, the focus may 
be lowest cost to obtain threshold requirements, 
superior performance within budget, superior stan¬ 
dardization for reduced logistics, maximum use of 
the open systems approach to reduce cost , or the 
like. A focus statement should: 


• Be a single objective to avoid confusion, 

• Be stated simply to avoid misinterpretation, and 

• Flave high-level support. 

Purpose: The purpose of the engineering effort 
should be described in general terms of the outputs, 
both end products and life-cycle enabling prod¬ 
ucts that are required. The stated purpose should 
answer the question, “What does the engineering 
effort have to produce?” 

Technical Strategy 

The basic purpose of a technical strategy is to link 
the development process with the acquisition or 
contract management process. It should include: 

• Development phasing and associated baselining, 

• Key engineering milestones to support risk 
management and business management mile¬ 
stones, 

• Associated parallel developments or product 
improvement considerations, and 

• Other management generated constraints or 
high-visibility activities that could affect the 
engineering development. 

Phasing and Milestones: The development 
phasing and baseline section should describe the 
approach to phasing the engineering effort, 
including tailoring of the basic process described 
in this book and a rationale for the tailoring. The 
key milestones should be in general keeping with 
the technical review process, but tailored as 
appropriate to support business management mile¬ 
stones and the project/program’s development 
phasing. Strategy considerations should also in¬ 
clude discussion of how design and verification 
will phase into production and fielding. This area 
should identify how production will be phased-in 
(including use of limited-rate initial production and 
long lead-time purchases), and that initial support 
considerations require significant coordination 
between the user and acquisition community. 
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Parallel Developments and Product Improve¬ 
ment: Parallel development programs necessary 
for the system to achieve its objectives should be 
identified and the relationship between the efforts 
explained. Any product improvement strategies 
should also be identified. Considerations such as 
evolutionary development and preplanned product 
improvement should be described in sufficient 
detail to show how they would phase into the 
overall effort. 

Impacts on Strategy 

All conditions or constraints that impact the strat¬ 
egy should be identified and the impact assessed. 
Key points to consider are: 

• Critical technologies development. 

• Cost As an Independent Variable (CAIV), and 

• Any business management directed constraint 
or activity that will have a significant influence 
on the strategy. 

Critical Technologies: Discussion of critical 
technology should include: 

• Risk associated with critical technology 
development and its impact on the strategy. 

• Relationship to baseline development, and 

• Potential impact on the overall development 
effort. 

Cost As an Independent Variable: Strategy con¬ 
siderations should include discussion of how 
CAIV will be implemented, and how it will impact 
the strategy. It should discuss how unit cost, de¬ 
velopment cost, life cycle cost, total ownership 
cost, and their interrelationships apply to the sys¬ 
tem development. This area should focus on how 
these costs will be balanced, how they will be con¬ 
trolled, and what impact they have on the strategy 
and design approach. 

Management Issues: Management issues that pose 
special concerns for the development strategy 


could cover a wide range of possible issues. In 
general, management issues identified as engineer¬ 
ing strategy issues are those that impact the ability 
to support the management strategy. Examples 
would include: 

• Need to combine developmental phases to 
accommodate management driven schedule or 
resource limitations, 

• Risk associated with a tight schedule or limited 
budget, 

• Contractual approach that increases technical 
risk, and 

• Others of a similar nature. 

Management-dictated technical activities—such as 
use of M&S, open systems, IPPD, and others— 
should not be included as a strategy issue unless 
they impact the overall systems engineering strat¬ 
egy to meet management expectations. The strat¬ 
egy discussion should lay out the plan, how it 
dovetails with the management strategy, and how 
management directives impact it. 

Systems Engineering Processes 

This area of the planning should focus on how the 
system engineering processes will be designed to 
support the strategy. It should include: 

• Specific methods and techniques used to 
perform the steps and loops of the systems en¬ 
gineering process, 

• Specific system analysis and control tools and 
how they will be used to support step and loop 
activities, and 

• Special design considerations that must be 
integrated into the engineering effort. 

Steps and Loops: The discussion of how the 
systems engineering process will be done should 
show the specific procedures and products that will 
ensure: 
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• Requirements are understood prior to the flow- 
down and allocation of requirements, 

• Functional descriptions are established before 
designs are formulated, 

• Designs are formulated that are traceable to 
requirements, 

• Methods exist to reconsider previous steps, and 

• Verification processes are in place to ensure that 
design solutions meet needs and requirements. 

This planning area should address each step and 
loop for each development phase, include identi¬ 
fication of the step-specific tools (Functional Flow 
Block Diagrams, Timeline Analysis, etc.) that will 
be used, and establish the verification approach. 
The verification discussion should identify all 
verification activities, the relationship to formal 
developmental T&E activities, and independent 
testing activities (such as operational testing). 

Norms of the particular technical area and the 
engineering processes of the command, agency, or 
company doing the tasks will greatly influence this 
area of planning. Flowever, whatever procedures, 
techniques, and analysis products or models used, 
they should be compatible with the basic principles 
of systems engineering management as described 
earlier in this book. 

An example of the type of issue this area would 
address is the requirements analysis during the 
system definition phase. Requirements analysis is 
more critical and a more central focus during sys¬ 
tem definition than in later phases. The establish¬ 
ment of the correct set of customer requirements 
at the beginning of the development effort is 
essential to proper development. Accordingly, the 
system definition phase requirements analysis 
demands tight control and an early review to verify 
the requirements are established well enough to 
begin the design effort. This process of control and 
verification necessary for the system definition 
phase should be specifically described as part of 


the overall requirements analysis process and 
procedures. 

Analysis and Control: Planning should identify 
those analysis tools that will be used to evaluate 
alternative approaches, analyze or assess effective¬ 
ness, and provide a rigorous quantitative basis for 
selecting performance, functional, and design 
requirements. These processes can include trade 
studies, market surveys. M&S, effectiveness analy¬ 
ses, design analyses, QFD, design of experiments, 
and others. 

Planning must identify the method by which 
control and feedback will be established and main¬ 
tained. The key to control is performance-based 
measurement guided by an event-based schedule. 
Entrance and exit criteria for the event-driven 
milestones should be established sufficient to 
demonstrate proper development progress has been 
completed. Event-based schedules and exit crite¬ 
ria are further discussed later in this chapter. 
Methods to maintain feedback and control are 
developed to monitor progress toward meeting the 
exit criteria. Common methods were discussed 
earlier in this book in the chapters on metrics, risk 
management, configuration management, and 
technical reviews. 

Design Considerations: In every system develop¬ 
ment there are usually technical activities that 
require special attention. These may come from 
management concerns, legal or regulatory direc¬ 
tives, social issues, or organizational initiatives. For 
example, a DoD program office will have to con¬ 
form to DoDD 5000.2-R, which lists several tech¬ 
nical activities that must be incoiporated into the 
development effort. DoD plans should specifically 
address each issue presented in the Program Design 
section of DoD 5000.2-R. 

In the case of a contractor there may be issues de¬ 
lineated in the contract, promised in the proposal, 
or established by management that the technical 
effort must address. The system engineering plan¬ 
ning must describe how each of these issues will 
be integrated into the development effort. 
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Organization 

Systems engineering management planning should 
identify the basic structure that will develop the 
system. Organizational planning should address 
how the integration of the different technical dis¬ 
ciplines, primary function managers, and other 
stakeholders will be achieved to develop the sys¬ 
tem. This planning area should describe how multi¬ 
disciplinary teaming would be implemented, that 
is, how the teams will be organized, tasked, and 
trained. A systems-level team should be established 
early to support this effort. Roles, authority, and 
basic responsibilities of the system-level design 
team should be specifically described. Establish¬ 
ing the design organization should be one of the 
initial tasks of the system-level design team. Their 
basic approach to organizing the effort should be 
described in the plan. Further information on 
organizing is contained in a later chapter. 

Resources 

The plan should identify the budget for the techni¬ 
cal development. The funds required should be 
matrixed against a calendar schedule based on the 
event-based schedule and the strategy. This should 
establish the basic development timeline with an 
associated high-level estimated spending profile. 
Shortfalls in funding or schedule should be ad¬ 
dressed and resolved by increasing funds, extend¬ 
ing schedule, or reducing requirements prior to the 
plan preparation. Remember that future analysis 
of development progress by management will tend 
to be based on this budget “promised” at plan 
inception. 

16.3 INTEGRATION OF PLANS - 
PROGRAM PLAN INTERFACES 

Systems engineering management planning must 
be coordinated with interfacing activities such as 
these: 

• Acquisition Strategy assures that technical plans 
take into account decisions reflected in the Ac¬ 
quisition Strategy. Conflicts must be identified 
early and resolved. 


• Financial plan assures resources match the 
needs in the tech plan. Conflicts should be 
identified early and resolved. 

• Test and Evaluation Master Plan (TEMP) as¬ 
sures it complements the verification approach. 
It should provide an integrated approach to 
verify that the design configuration will meet 
customer requirements. This approach should 
be compatible with the verification approach 
delineated in the systems engineering plan. 

• Configuration management plan assures that the 
development process will maintain the system 
baselines and control changes to them. 

• Design plans (e.g., electrical, mechanical, struc¬ 
tural, etc.) coordinates identification of IPT 
team composition. 

• Integrated logistics support planning and sup¬ 
port analysis coordinates total system support. 

• Production/Manufacturing plan to coordinate 
activities concerning design producibility, and 
follow-on production, 

• Quality management planning assures that 
quality engineering activities and quality man¬ 
agement functions are included in system 
engineering planning, 

• Risk management planning establishes and 
coordinates technical risk management to 
support total program risk management. 

• Interoperability planning assures interopera¬ 
bility suitability issues are coordinated with sys¬ 
tem engineering planning. (Where interop¬ 
erability is an especially critical requirement 
such as, communication or information systems, 
it should be addressed as a separate issue with 
separate integrated teams, monitoring, and 
controls). 

• Others such as M&S plan, software develop¬ 
ment plan, human integration plan, environ¬ 
ment, safety and health planning, also interface. 
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Things to Watch 

A well developed technical management plan will 

include: 

• The expected benefit to the user, 

• How a total systems development will be 
achieved using a systems engineering approach, 

• How the technical plan complements and sup¬ 
ports the acquisition or management business 
plan, 

• How incremental reviews will assure that the 
development stays on track, 

• How costs will be reduced and controlled, 

• What technical activities are required and who 
will perform them, 

• How the technical activities relate to work 
accomplishment and calendar dates, 

• How system configuration and risk will be 
controlled, 

• How system integration will be achieved, 


• How the concerns of the eight primary life cycle 
functions will be satisfied, 

• How regulatory and contractual requirements 
will be achieved, and 

• The feasibility of the plan, i.e., is the plan 
practical and executable from a technical, 
schedule, and cost perspective. 

16.4 SUMMARY POINTS 

• Systems engineering planning should establish 
the organizational structure that will achieve the 
engineering objectives. 

• Planning must include event-based scheduling 
and establish feedback and control methods. 

• It should result in important planning and 
control documents for carrying out the 
engineering effort. 

• It should identify the estimated funding and 
detail schedule necessary to achieve the strategy. 

• Systems engineering planning should establish 
the proper relationship between the acquisition 
and technical processes. 
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APPENDIX 16-A 

SCHEDULES 


The event-based schedule, sometimes referred to 
as the Systems Engineering Master Schedule 
(SEMS) or Integrated Master Schedule (IMS) is a 
technical event-driven (not time-driven) plan pri¬ 
marily concerned with product and process 
development. It forms the basis for schedule con¬ 
trol and progress measurement, and relates 
engineering management events and accomplish¬ 
ments to the WBS. These events are identified 
either in the format of entry and exit events (e.g. 
initiate PDR, complete PDR) or by using entry and 
exit criteria for each event. Example exit criteria 
shown in Figures 16-1 and 16-2. 


The program office develops an event-based 
schedule that represents the overall development 
effort. This schedule is usually high-level and 
focused on the completion of events that support 
the acquisition milestone decision process. An 
event-based schedule is developed by the contrac¬ 
tor to include significant accomplishments that 
must be completed in order to meet the progress 
required prior to contract established events. The 
contractor also includes events, accomplishments, 
and associated success criteria specifically identi¬ 
fied by the contract. DoD program offices can use 
the contractor’s event-based schedule and the 


System Requirements 
Review (SRR) 

System Functional 
Review/Software Spec 
Review(SFR/SSR) 

Preliminary Design 

Review (PDR) 

• Mission Analysis completed 

• Support Strategy defined 

• System options decisions 
completed 

• Design usage defined 

• Operational performance 
requirement defined 

• Manpower sensitivities 
completed 

• Operational architecture 
available and reviewed 

• Installed environments defined 

• Maintenance concept defined 

• Preliminary design criteria 
established 

• Preliminary design margins 
established 

• Interfaces defined/preliminary 
interface specs completed 

• Software and software support 
requirements completed 

• Baseline support/resources 
requirements defined 

• Support equipment capability 
defined 

• Technical architecture prepared 

• System defined and requirements 
shown to be achievable 

• Design analyses/definition 
completed 

• Material/parts characterization 
completed 

• Design maintainability analysis 
completed/support requirements 
defined 

• Preliminary production plan 
completed 

• Make/buy decisions finalized 

• Breadboard investigations 
completed 

• Coupon testing completed 

• Design margins completed 

• Preliminary FMECA completed 

• Software functions and architec¬ 
ture and support defined 

• Maintenance tasks trade studies 
completed 

• Support equipment development 
specs completed 


Figure 16-1. Sample Event-Based Schedule Exit Criteria 


153 












Systems Engineering Fundamentals 


Chapter 16 


Critical Design Review 

Test Readiness Review 
(CDR/TRR) 

System Verfication Review/ 
Functional Configuration Audit 
(SVR/FCA) 

Physical Configuration Audit 
(PCA) 

• Parts, materials, processes 
selected 

• Development tests completed 

• Inspection points/criteria 
completed 

• Component level FMECA 
completed 

• Repair level analysis completed 

• Facility requirements defined 

• Software test descriptions 
completed 

• Flardware and software hazard 
analysis completed 

• Firmware spt completed 

• Software programmers manual 
completed 

• Durability test completed 

• Maintinability analyses com¬ 
pleted 

• Qualification test procedures 
approved 

• Producibility analyses com¬ 
pleted 

• All verification tasks completed 

• Durability tests completed 

• Long lead time items identified 

• PME and operational training 
completed 

• Tech manuals completed 

• Flight test plan approved 

• Support and training equipment 
developed 

• Fielding analysis completed 

• Provisioning data verified 

• Qualification testing completed 

• All QA provisions finalized 

• All manufacturing process 
requirements and documenta¬ 
tion finalized 

• Product fabrication specifica¬ 
tions finalized 

• Support and training equipment 
qualification completed 

• All acceptance test require¬ 
ments completed 

• Life management plan com¬ 
pleted 

• System support capability 
demonstrated 

• Post production support 
analysis completed 

• Final software description 
document and all user manuals 
complete 


Figure 16-2. Sample Event-Driven Schedule Exit Criteria (continued) 


contractor’s conformance to it for several purposes: 
source selection, monitoring contractor progress, 
technical and other reviews, readiness for option 
award, incentives/awards determination, progress 
payments decision, and similar activities. 

The event-based schedule establishes the key 
parameters for determining the progress of a 
development program. To some extent it controls 
and interfaces with systems engineering manage¬ 
ment planning, integrated master schedules and in¬ 
tegrated master plans, as well as risk management 
planning, system test planning, and other key plans 
which govern the details of program management. 


The calendar - or detail schedule is a time-based 
schedule that shows how work efforts will support 
tasks and events identified in the event-based 
schedule. It aligns the tasks and calendar - dates to 
show when each significant accomplishment must 
be achieved. It is a key component for developing 
Earned Value metrics. The calendar schedule is 
commonly referred to as the detail schedule, 
systems engineering detail schedule, or SEDS. The 
contractor is usually required to maintain the 
relationship between the event and calendar 
schedules for contract required activities. Figure 
16-3 shows the relationship between the system 
requirements, the WBS, the contractual require¬ 
ments, the event-based schedule, and the detail 
schedule. 
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Figure 16-3. Event-Based—Detailed Schedule Interrelationships 


Schedule Summary 

The event-based schedule establishes the key tasks 
and results expected. The event-based schedule 
establishes the basis for a valid calendar-based 
(detail) schedule. 
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PRODUCT IMPROVEMENT 
STRATEGIES 


17.1 INTRODUCTION 

Complex systems do not usually have stagnant 
configurations. A need for a change during a 
system’s life cycle can come from many sources 
and effect the configuration in infinite ways. The 
problem with these changes is that, in most cases 
it is difficult, if not impossible, to predict the na¬ 
ture and timing of these changes at the beginning 
of system development. Accordingly, strategics or 
design approaches have been developed to reduce 
the risk associated with predicted and unknown 
changes. 

Well thought-out improvement strategies can help 
control difficult engineering problems related to: 

• Requirements that are not completely under¬ 
stood at program start, 

• Technology development that will take longer 
than the majority of the system development, 

• Customer needs (such as the need to combat a 
new military threat) that have increased, been 
upgraded, are different, or are in flux, 

• Requirements change due to modified policy, 
operational philosophy, logistics support phi¬ 
losophy, or other planning or practices from the 
eight primary life cycle function groups, 

• Technology availability that allows the system 
to perform better and/or less expensively, 

• Potential reliability and maintainability up¬ 
grades that make it less expensive to use, 
maintain, or support, including development of 
new supply support sources, 


• Safety issues requiring replacement of unsafe 
components, and 

• Service life extension programs that refurbish 
and upgrade systems to increase then - service life. 

In DoD, the 21st century challenge will be improv¬ 
ing existing products and designing new ones that 
can be easily improved. With the average service 
life of a weapons system in the area of 40 or more 
years, it is necessary that systems be developed 
with an appreciation for future requirements, fore¬ 
seen and unforeseen. These future requirements 
will present themselves as needed upgrades to 
safety, performance, supportability, interface com¬ 
patibility, or interoperability; changes to reduce 
cost of ownership; or major rebuild. Providing 
these needed improvements or corrections form 
the majority of the systems engineer’s post¬ 
production activities. 

17.2 PRODUCT IMPROVEMENT 
STRATEGIES 

As shown by Figure 17-1, these strategies vary 
based on where in the life cycle they are applied. 
The strategies or design approaches that reflect 
these improvement needs can be categorized as 
planned improvements, changes in design or 
production, and deployed system upgrades. 

Planned Improvements 

Planned improvements strategies include evolu¬ 
tionary acquisition, preplanned product develop¬ 
ment, and open systems. These strategies are not 
exclusive and can be combined synergistically in 
a program development. 
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Integrated Inputs of All Functional Disciplines 


Figure 17-1. Types of Product Improvement Strategies 
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• General for the System 
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“The lack of specificity 
and detail in identifying the final 
system capability is what 
distinguishes Evolutionary 
Acquisition from an 
acquisition strategy based 
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Figure 17-2. Evolutionary Acquisition 
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Evolutionary Acquisition: Evolutionary acquisi¬ 
tion is the preferred approach to systems acquisi¬ 
tion in DoD. In an environment where technology 
is a fast moving target and the key to military su¬ 
periority is a technically superior force, the require¬ 
ment is to transition useful capability from devel¬ 
opment to the user as quickly as possible, while 
laying the foundation for further changes to occur 
at later dates. Evolutionary acquisition is an ap¬ 
proach that defines requirements for a core capa¬ 
bility, with the understanding that the core is to be 
augmented and built upon (evolved) until the sys¬ 
tem meets the full spectrum of user requirements. 
The core capability is defined as a function of user 
need, technology maturity, threat, and budget. The 
core is then expanded as need evolves and the other 
factors mentioned permit. 

A key to achieving evolutionary acquisition is the 
use of time-phased requirements and continuous 
communication with the eventual user, so that re¬ 
quirements are staged to be satisfied incrementally, 


rather than in the traditional single grand design 
approach. Planning for evolutionary acquisition 
also demands that engineering designs be based 
on open system, modular design concepts that per¬ 
mit additional increments to be added over time 
without having to completely re-design and re¬ 
develop those portions of the system already 
fielded. Open designs will facilitate access to recent 
changes in technologies and will also assist in con¬ 
trolling costs by taking advantage of commercial 
competition in the marketplace. This concept is 
not new; it has been employed for years in the 
C4ISR community, where system are often in 
evolution over the entire span of their lifecycles. 

Preplanned Product Improvement (P3I): Often 
referred to as P3I, preplanned product improve¬ 
ment is an appropriate strategy when requirements 
are known and firm, but where constraints (typi¬ 
cally either technology or budget) make some 
portion of the system unachievable within the 
schedule required. If it is concluded that a militarily 



Responsive to threat changes 
Accommodates future technology 
IOC can be earlier 
Reduced development risk 
Possible subsystem competition 
Increased effective operational life 




The P3I acquisition 
management challenge is to acquire 
systems with interfaces and accessibility 
as an integral part of the design so that 
the deferred element(s) can be 
incorporated in a cost-effective manner 

lAihan that/ hoo/vno oi/oi/oh/a 



Acquisition Issues 

Longer Range Planning 
Parallel Efforts 

Standards and Interface Capacity 
Modular Equipment/Open Systems 


Increased initial development cost 

Increased technical requirements 

complexity 

More complex CM 

Sensitive to funding streams 

Parallel development management 




Figure 17-3. Pre-Planned Product Improvement 
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useful capability can be fielded as an interim solu¬ 
tion while the portion yet to be proceeds through 
development, then P3I is appropriate. The approach 
generally is to handle the improvement as a sepa¬ 
rate, parallel development; initially test and deliver 
the system without the improvement; and prove 
and provide the enhanced capability as it becomes 
available. The key to a successful P3I is the estab¬ 
lishment of well-defined interface requirements for 
the system and the improvement. Use of a P3I will 
tend to increase initial cost, configuration 
management activity, and technical complexity. 
Figure 17-3 shows some of the considerations in 
deciding when it is appropriate. 

Open Systems Approach: The open system design 
approach uses interface management to build flex¬ 
ible design interfaces that accommodate use of 
competitive commercial products and provide 
enhanced capacity for future change. It can be used 
to prepare for future needs when technology is yet 
not available, whether the operational need is 
known or unknown. The open systems focus is to 
design the system such that it is easy to modify 
using standard interfaces, modularity, recognized 
interface standards, standard components with 
recognized common interfaces, commercial and 
nondevelopmental items, and compartmentalized 
design. Open system approaches to design are 
further discussed at the end of this chapter. 

Changes in Design or Production 

Engineering Change Proposals (ECPs): Changes 
that are to be implemented during the development 
and production of a given system are typically ini¬ 
tiated through the use of ECPs. If the proposed 
change is approved (usually by a configuration 
control board) the changes to the documentation 
that describes the system are handled by formal 
configuration management, since, by definition, 
ECPs, when approved, change an approved base¬ 
line. ECPs govern the scope and details of these 
changes. ECPs may address a variety of needs, 
including correction of deficiencies, cost reduc¬ 
tion, and safety. Furthermore, ECPs may been as¬ 
signed differing levels of priority from routine to 
emergency. MIL-HDBK-61, Configuration Man¬ 
agement Guidance, offers an excellent source of 


advice on issues related to configuration changes. 

Block Change before Deployment: Block changes 
represent an attempt to improve configuration 
management by having a number of changes 
grouped and applied such that they will apply con¬ 
sistently to groups (or blocks) of production items. 
This improves the management and configuration 
control of similar items substantially in compari¬ 
son to change that is implemented item by item 
and single change order by single change order. 
When block changes occur, the life cycle impact 
should be carefully addressed. Significant differ¬ 
ences in block configurations can lead to different 
manuals, supply documentation, training, and 
restrictions as to locations or activities where the 
system can be assigned. 

Deployed Systems Upgrades 

Major Rebuild: A major rebuild results from the 
need for a system that satisfies requirements sig¬ 
nificantly different or increased from the existing 
system, or a need to extend the life of a system 
that is reaching the end of its usable life. In both 
cases the system will have upgraded requirements 
and should be heated as basically a new system 
development. A new development process should 
be started to establish and control configuration 
baselines for the rebuilt system based on the 
updated requirements. 

Major rebuilds include remanufacturing, service- 
life extension programs, and system developments 
where significant parts of a previous system will 
be reused. Though rebuilding existing systems can 
dramatically reduce the cost of a new system in 
some cases, the economies of rebuild can be 
deceiving, and the choice of whether to pursue a 
rebuild should be done after careful use of trade 
studies. The key to engineering such systems is to 
remember that they are new systems and require 
the full developmental considerations of baselin¬ 
ing, the systems engineering process, and life cycle 
integration. 

Post-Production Improvement: In general, product 
improvements become necessary to improve the 
system or to maintain the system as its components 
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reach obsolescence. These projects generally re¬ 
sult in a capability improvement, but for all practi¬ 
cal puiposes the system still the serves the same 
basic need. These improvements are usually char¬ 
acterized by an upgrade to a component or sub¬ 
system as opposed to a total system upgrade. 

Block Upgrades: Post-production block upgrades 
are improvements to a specific group of the system 
population that provides a consistent configura¬ 
tion within that group. Block upgrades in post¬ 
production serve the same general purpose of 
controlling individual system configurations as 
production block upgrades, and they require the 
same level of life-cycle integration. 

Modifying an Existing System 

Upgrading an existing system is a matter of fol¬ 
lowing the system engineering process, with an 
emphasis on configuration and interface manage¬ 
ment. The following activities should be included 
when upgrading a system: 

• Benchmark the modified requirements both for 
the upgrade and the system as a whole, 

• Perform functional analysis and allocation on 
the modified requirements, 

• Assess the actual capability of the pre-upgrade 
system, 

• Identify cost and risk factors and monitor them, 

• Develop and evaluate modified system alterna¬ 
tives, 

• Prototype the chosen improvement alternative, 
and 

• Verify the improvement. 

Product improvement requires special attention 
to configuration and interface management. It 
is not uncommon that the existing system’s con¬ 
figuration will not be consistent with the existing 
configuration data. Form, fit, and especially func¬ 
tion interfaces often represent design constraints 


that are not always readily apparent at the outset 
of a system upgrade. Upgrade planning should 
ensure that the revised components will be com¬ 
patible at the interfaces. Where interfaces are im¬ 
pacted, broad coordination and agreement is nor¬ 
mally required. 

Traps in Upgrading Deployed Systems 

When upgrading a deployed system pay attention 
to the following significant traps: 

Scheduling to minimize operational impacts: The 
user’s operational commitments will dictate the 
availability of the system for modification. If the 
schedule conflicts with an existing or emerging 
operational need, the system will probably not 
become available for modification at the time 
agreed to. Planning and contractual arrangements 
must be flexible enough to accept unforeseen sche¬ 
dule changes to accommodate user’s unanticipated 
needs. 

Configuration and interface management: Con¬ 
figuration management must address three configu¬ 
rations: the actual existing configuration, the modi¬ 
fication configuration, and the final system con¬ 
figuration. The key to successful modification is 
the level of understanding and control associated 
with the interfaces. 

Logistics compatibility problems: Modification 
will change the configuration, which in most cases 
will change the supply support and maintenance 
considerations. Coordination with the logistics 
community is essential to the long-term operational 
success of the modification. 

Minimal resources available: Modifications tend 
to be viewed as simple changes. As this chapter 
has pointed out, they are not; and they should be 
carefully planned. That planning should include 
an estimate of needed resources. If the resources 
are not available, either the project should be 
abandoned, or a plan formulated to mitigate and 
control the risk of an initial, minimal budget com¬ 
bined with a plan for obtaining additional 
resources. 
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Funding restrictions ($ color) drive the need to separate 
performance increase from supportability changes 


If... 


Fund 

development 
and test 
with... 


Fund mod 
kit with... 


Fund 

installation 

with... 



Product improvement planning must be driven by 
risk management, not by $ color or calendar! 


Figure 17-4. Funding Rule for DoD System Upgrades 


Limited competitors: Older systems may have only 
a few suppliers that have a corporate knowledge 
of the particular system functions and design. This 
is especially problematic if the original system 
components were commercial or NDIs that the de¬ 
signer does not have product baseline data for. In 
cases such as these, there is a learning process that 
must take place before the designer or vendor can 
adequately support the modification effort. De¬ 
pending on the specific system, this could be a 
major effort. This issue should be considered very 
early in the modification process because it has 
serious cost implications. 

Government funding rides: As Figure 17-4 shows 
the use of government funding to perform system 
upgrades has restrictions. The puipose of the up¬ 
grade must be clear and justified in the planning 
efforts. 


17.3 ROLES AND RESPONSIBILITIES 

Modification management is normally a joint gov¬ 
ernment and contractor responsibility. Though any 


specific system upgrade will have relationships 
established by the conditions surrounding the par¬ 
ticular program, government responsibilities would 
usually include: 

• Providing a clear statement of system require¬ 
ments, 

• Planning related to government functions, 

• Managing external interfaces, 

• Managing the functional baseline configuration, 
and 

• Verifying that requirements are satisfied. 

Contractor responsibilities are established by the 
contract, but would normally include: 

• Technical planning related to execution, 

• Defining the new performance envelope, 

• Designing and developing modifications, and 
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• Providing evidence that changes made have 
modified the system as required. 

System Engineering Role 

The systems engineering role in product improve¬ 
ment includes: 

• Planning for system change, 

• Applying the systems engineering process, 

• Managing interface changes, 

• Identifying and using interface standards which 
facilitate continuing change, 

• Ensuring life cycle management is implemented, 

• Monitoring the need for system modifications, 
and 


• Ensuring operations, support activities, and 
early field results are considered in planning. 

17.4 SUMMARY POINTS 

• Complex systems do not usually have stagnant 
configurations. 

• Planned improvements strategies include 
evolutionary acquisition, preplanned product 
development, and open systems. 

• A major rebuild should be treated as a new 
system development. 

• Upgrading an existing system is a matter of 
following the system engineering process, with 
an emphasis on configuration and interface 
management. 

• Pay attention to the traps. Upgrade projects have 
many. 
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SUPPLEMENT 17-A 

OPEN SYSTEM APPROACH 


The open system approach is a business and 
technical approach to system development that 
results in systems that are easier to change or 
upgrade by component replacement. It is a system 
development logic that emphasizes flexible 
interfaces and maximum interoperability, optimum 
use of commercial competitive products, and 
enhanced system capacity for future upgrade. The 
value of this approach is that open systems have 
flexibility, and that flexibility translates into ben¬ 
efits that can be recognized from business, 
management, and technical perspectives. 

From a management and business view, the open 
system approach directs resources to a more in¬ 
tensive design effort with the expectation of a life 
cycle cost reduction. As a business approach it 
supports the DoD policy initiatives of CAIV, in¬ 
creased competition, and use of commercial prod¬ 
ucts. It is a technical approach that emphasizes 


systems engineering, interface control, modular 
design, and design for upgrade. As a technical ap¬ 
proach it supports the engineering goals of design 
flexibility, risk reduction, configuration control, 
long-term supportability, and enhanced utility. 

Open Systems Initiative 

In DoD the open system initiative was begun as a 
result of dramatic changes in the computer indus¬ 
try that afforded significant advantages to design 
of C4ISR and IT systems. The standardization 
achieved by the computer industry allows C4ISR 
and IT systems to be designed using interface 
standards to select off-the-shelf components to 
form the system. This is achieved by using 
commercially-supported specifications and 
standards for specifying system interfaces (exter¬ 
nal and internal, functional and physical), prod¬ 
ucts, practices, and tools. An open system is one 



Figure 17-5. C4I and IT Development 
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Figure 17-6. Simplified Computer Resource Reference Model 
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in which interfaces are fully described by open 
standards. 1 An open system approach extends this 
concept further by using modular design and 
interface design to enhance the availability of mul¬ 
tiple design solutions, especially those reflecting 
use of open standards, competitive commercial 
components, NDIs, and future upgrade capability. 

As developed in the C4ISR and IT communities, 
the open system approach requires the design of 
three architectures: operational, technical, and 
system. 

As shown in Figure 17-5, the first one prepared is 
an operational architecture that defines the tasks, 
operational elements, and information flows 
required to accomplish or support an operational 
function. The user community generates the 
operational concepts that form an operational 
architecture. The operational architecture is 
allusive. It is not a specific document required to 
be developed by the user such as the ORD; but 


because of their operational nature, the user must 
provide the components of the operational 
architecture. It is usually left to the developer to 
assemble and structure the information as part of 
the system definition requirements analysis. Once 
the operational architecture has clearly defined the 
operational need, development of a system 
architecture 2 is begun. 

The (open) system architecture is a set of descrip¬ 
tions, including graphics, of systems and intercon¬ 
nections supporting the operational functions 
described in the operational architecture. Early in 
the (open) system architecture development a 
technical architecture is prepared to establish a set 
of rules, derived from open consensus-based 
industry standards, to govern the arrangement, 
interaction, and interdependence of the elements 
of a reference model. Reference models are a com¬ 
mon conceptual framework for the type of system 
being designed. (A simple version for computer 
resources is shown in Figure 17-6.) 


1 Open Standards are non-proprietary, consensus-based standards widely accepted by industry. Examples include S AE, IEEE, and ISO 
standards. 

2 This system architecture typically describes the end product but not the enabling products. It relies heavily on interface definitions to 
describe system components. 
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The technical architecture identifies the services, 
interfaces, standards, and their relationships; and 
provides the technical guidelines upon which 
engineering specifications are based, common 
building blocks are built, and product lines are 
developed. In short, the technical architecture be¬ 
comes a design requirement for developing the 
system. (The purpose, form, and function of the 
technical architecture is similar to building codes.) 

The system architecture is then further developed 
to eventually specify component performance and 
interface requirements. These are then used to 
select the specific commercial components that 
form the system under development. This process, 
called an implementation, envisions the produc¬ 
tion process as consisting primarily of selecting 
components, conformance (to the interface and 
performance requirements) management, and 
assembly, with little or no need for detailed design 
fabrications. 

The process described above has allowed signifi¬ 
cant achievements in computer-related develop¬ 
ments. Other technical fields have also used the 
open system design approach extensively. (Com¬ 
mon examples are the electrical outlets in your 
home and the tire-to-wheel interface on your car). 
In most cases the process is not as well defined as 
it is in the current digital electronics area. A con¬ 
sistent successful use of the open design concept, 
in and outside the electronics field, requires an 
understanding of how this process relates to the 
activities associated with systems engineering 
management. 

Systems Engineering Management 

The open system approach impacts all three 
essential elements of systems engineering manage¬ 
ment: systems engineering phasing, the systems 
engineering process, and life cycle considerations. 
It requires enhanced interface management in the 
systems engineering process, and requires specific 
design products be developed prior to engineer¬ 
ing-event milestones. The open systems approach 
is inherently life-cycle friendly. It favorably 
impacts production and support functions, but it 


also requires additional effort to assure life-cycle 
conformance to interface requirements. 

Open Systems Products and 
SE Development Phasing 

A system is developed with stepped phases that 
allow an understanding of the operational need to 
eventually evolve into a design solution. Though 
some tailoring of this concept is appropriate, the 
basic phasing (based on the operational concept 
preceding the system description, which precedes 
the preliminary design, which precedes the detailed 
design) is necessary to coordinate the overall 
design process and control the requirements flow- 
down. As shown by Figure 17-7 the open system 
approach blends well with these development 
phases. 

Concept Studies Phase 

The initial detailed operational concept, including 
operational architectures, should be a user-com¬ 
munity output (with some acquisition engineering 
assistance) produced during the concept explora¬ 
tion phase that emphasizes operational concepts 
associated with various material solutions. The 
operational concept is then updated as necessary 
for each following phase. Analysis of the initial 
operational concept should be a key element of 
the operational view output of the system defini¬ 
tion phase requirements analysis. An operational 
architecture developed for supporting the system 
description should be complete, comprehensive, 
and clear; and verified to be so at the Alternative 
Systems Review. If the operational architecture 
cannot be completed, then a core operational 
capability must be developed to establish the basis 
for further development. Where a core capability 
is used, core requirements should be complete and 
firm, and the process for adding expanded 
requirements should be clear and controlled. 

System Definition Phase 

System interface definitions, such as the technical 
architecture, and high-level (open) system archi¬ 
tecture should be complete in initial form at the 
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Figure 17-7. Phasing of Open System Development 


end of the system definition phase (along with other 
functional baseline documentation). Successful 
completion of these items is required to perform 
the preliminary design, and they should be avail¬ 
able for the System Functional Review, also 
referred to as the System Definition Review or Sys¬ 
tem Design Review. The open system documenta¬ 
tion can be separate or incorporated in other func¬ 
tional baseline documentation. The criteria for 
acceptance should be established in the systems 
engineering management plan as phase-exit 
criteria. 

Preliminary Design Phase 

Along with other allocated baseline documenta¬ 
tion, the interface definitions should be updated 
and the open-system architecture completed by the 
end of the preliminary design effort. This docu¬ 
mentation should also identify the proper level of 
openness (that is, the level of system decomposi¬ 
tion at which the open interfaces are established) 
to obtain the maximum cost and logistic advantage 
available from industry practice. 


The preliminary design establishes performance- 
based descriptions of the system components, as 
well as the interface and structure designs that 
integrate those components. It is in this phase that 
the open system approach has the most impact. 
Interface control should be enhanced and focused 
on developing modular designs that allow for maxi¬ 
mum interchange of competitive commercial prod¬ 
ucts. Review of the technical architecture (or in¬ 
terface definitions) becomes a key element of re¬ 
quirements analysis, open system focused func¬ 
tional partitioning becomes a key element of func¬ 
tional analysis and allocation, iterative analysis of 
modular designs becomes a key element of design 
synthesis, and conformance management becomes 
a key element of verification. Open system related 
products, such as the technical architecture, inter¬ 
face management documentation, and conform¬ 
ance management documentation, should be key 
data reviewed at the Preliminary Design Review. 
Again, the criteria for acceptance should be estab¬ 
lished in the systems engineering management plan 
as phase-exit criteria. 
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Detail Design Phase 

The detail design phase becomes the implementa¬ 
tion for those parts of the system that have achieved 
open system status. Conformance management 
becomes a significant activity as commercial com¬ 
ponents are chosen to meet performance and 
interface requirements. Conformance and interface 
design testing becomes a driving activity during 
verification to assure an open system or subsystem 
has been achieved and that components selected 
meet interface requirements and/or standards. 

Systems Engineering Process 

The systems engineering problem solving process 
consists of process steps and loops supported by 
system analysis and control tools. The focus of the 
open systems engineering process is compartmen¬ 
talized design, flexible interfaces, recognized in¬ 
terface standards, standard components with 
recognized common interfaces, use of commercial 
and NDIs, and an increased emphasis on interface 
control. As shown by Figure 17-8, the open-sys¬ 
tem approach complements the systems engineer¬ 
ing process to provide an upgradeable design. 


Requirements analysis includes the review and 
update of interface standards and other interface 
definitions generated as output from previous 
systems engineering processes. Functional analy¬ 
sis and allocation focuses on functional partition¬ 
ing to identify functions that can be performed in¬ 
dependent of each other in order to minimize func¬ 
tional interfaces. Design synthesis focuses on 
modular design with open interfaces, use of open 
standards compliant commercial products, and the 
development of performance and interface speci¬ 
fications. The verification processes include con¬ 
formance testing to validate the interface require¬ 
ments are appropriate and to verify components 
chosen to implement the design meet the interface 
requirements. Engineering open designs, then, does 
not alter the fundamental practices within systems 
engineering, but, rather, provides a specific focus 
to the activities within that process. 

System Engineering Control: 

Interface Management 

The key to the open systems engineering process 
is interface management. Interface management 
should be done in a more formal and comprehen¬ 
sive manner to rigidly identify all interfaces and 
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Figure 17-8. Open System Approach to the Systems Engineering Process 
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control the flowdown and integration of interface 
requirements. The interfaces become controlled 
elements of the baseline equal to (or considered 
part of) the configuration. Open system interface 
management emphasizes the correlation of inter¬ 
face requirements between interfacing systems. 
(Do those designing the interfacing systems 
understand the interface requirements in the same 
way?) Computer-Aided System Engineering 
(CASE) generated schematic block diagrams can 
be used to Pack interface design activity. 

An open system is also characterized by multiple 
design solutions within the interfaces with empha¬ 
sis on leveraging best commercial practice. The 
interface management effort must control interface 
design such that interfaces specifically chosen for 
an open system approach are designed based on 
the following priority: 

• Open standards that allow competitive products, 

• Open interface design that allows installation 
of competitive products with minimal change, 

• Open interface design that allows minimal 
change installation of commercial or NDI prod¬ 
ucts currently or planned to be in DoD use, and 
last, 

• Unique design with interfaces designed with 
upgrade issues considered. 

Note that these are clear priorities, not options. 

Level of Openness 

The level at which the interface design should focus 
on openness is also a consideration. Each system 
may have several levels of openness depending on 
the complexity of the system and the differences 
in the technology within the system. The level cho¬ 
sen to define the open interfaces should be 
supported by industry and be consistent with 
program objectives. For example, for most digital 
electronics that level is the line-replaceable (LRU) 
and shop-replaceable (SRU) level. On the other 
hand the Joint Strike Fighter intends to establish 
openness at a very high subsystem level to achieve 


a major program objective, development of 
different planes using common building blocks 
(which, in essence, serve as the reference model 
for the family of aircraft). The open system ap¬ 
proach designed segments of a larger system could 
have additional openness at a lower level. For ex¬ 
ample, the Advanced Amphibious Assault Vehicle 
(AAAV) engine compartment is an open approach 
design allowing for different engine installation 
and future upgrade capability. On a lower level 
within the compartment the fuel filters, lines, and 
connectors are defined by open standard based 
interfaces. Other systems will define openness at 
other levels. Program objectives (such as inter¬ 
operability, upgrade capability, cost-effective sup¬ 
port, affordability, and risk reduction) and industry 
practice (based on market research) drive the 
choice of the level of openness that will best assure 
optimum utility and availability of the open system 
approach. 

Life Cycle Considerations 

Life cycle integration is established primarily 
through the use of integrated teaming that com¬ 
bines the design and life cycle planning. The ma¬ 
jor impacts on life-cycle activity include: 

• Time and cost to upgrade a system is reduced. 
It is common in defense systems, which have 
average life spans in excess of 40 years, that 
they will require upgrade in their life due to 
obsolescence of original components, threat 
increase, and technology push that increases 
economy or performance. (Most commercial 
products are designed for a significantly shorter 
life than military systems, and designs that rely 
on these commercial products must expect that 
original commercial components will not 
necessarily be available throughout the system’s 
life cycle.) By using an open system approach 
the ability to upgrade a system by changing a 
single or set of components is greatly enhanced. 
In addition, the open system approach eases the 
design problem of replacing the component, 
thereby reducing the cost and schedule of up¬ 
grade, which in turn reduces the operational 
impact. 
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• An open system approach enhances the use 
of competitive products to support the system. 
This flexibility tends to reduce the cost associ¬ 
ated with supply support, but more importantly 
improves component and parts availability. 

• Conformance management becomes a part of 
the life cycle configuration process. Replace¬ 
ment of components in an open system must 
be more controlled because the government has 
to control the system configuration without 
controlling the detail component configuration 
(which will come from multiple sources, all 
with different detail configurations). The gov¬ 
ernment must expect that commercial suppli¬ 
ers will control the design of their components 
without regard to the government’s systems. 
The government therefore must use perfor¬ 
mance- and interface-based specifications to 
assure the component will provide service 
equivalent to that approved through the acqui¬ 
sition process. Conformance management is the 


process that tracks the interface requirements 
through the life cycle, and assures that the new 
product meets those requirements. 

Summary Comments 

Open system design is not only compatible with 
systems engineering; it represents an approach that 
enhances the overall systems engineering effort. It 
controls interfaces comprehensively, provides in¬ 
terface visibility, reduces risk through multiple 
design solutions, and insists on life cycle interface 
control. This emphasis on interface identification 
and control improves systems engineers’ capability 
to integrate the system, probably one of the bal d¬ 
est jobs they have. It also improves the tracking of 
interface requirements flow down, another key job 
of the systems engineer. Perhaps most importantly, 
this rigorous interface management improves sys¬ 
tems engineers’ ability to correctly determine 
where commercial items can be properly used. 
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18.1 INTEGRATED DEVELOPMENT 

DoD has, for years, required that system designs 
be integrated to balance the conflicting pressure 
of competing requirements such as performance, 
cost, supportability, producibility, and testability. 
The use of multi-disciplinary teams is the approach 
that both DoD and industry increasing have taken 
to achieve integrated designs. Teams have been 
found to facilitate meeting cost, performance, and 
other objectives from product concept through 
disposal. 

The use of multi-disciplinary teams in design is 
known as Integrated Product and Process Devel¬ 
opment, simultaneous engineering, concurrent 
engineering. Integrated Product Development, 
Design-Build, and other proprietary and non-pro¬ 
prietary names expressing the same concept. (The 
DoD use of the term Integrated Product and Pro¬ 
cess Development (IPPD) is a wider concept that 
includes the systems engineering effort as an ele¬ 
ment. The DoD policy is explained later in this 
chapter.) Whatever name is used, the fundamental 
idea involves multi-functional, integrated teams 
(preferably co-located), that jointly derive require¬ 
ments and schedules that place equal emphasis on 
product and process development. The integration 
requires: 

• Inclusion of the eight primary functions in the 
team(s) involved in the design process, 

• Technical process specialties such as quality, 
risk management, safety, etc., and 

• Business processes (usually in an advisory 
capacity) such as, finance, legal, contracts, and 
other non-technical support. 


Benefits 

The expected benefits from team-based integration 
include: 

• Reduced rework in design, manufacturing, 
planning, tooling, etc., 

• Improved first time quality and reduction of 
product variability, 

• Reduced cost and cycle time, 

• Reduced risk, 

• Improved operation and support, and 

• General improvement in customer satisfaction 
and product quality throughout its life cycle. 

Characteristics 

The key attributes that characterize a well 
integrated effort include: 

• Customer focus, 

• Concurrent development of products and 
processes, 

• Early and continuous life cycle planning, 

• Maximum flexibility for optimization, 

• Robust design and improved process capability, 

• Event-driven scheduling, 

• Multi-disciplinary teamwork. 
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• Empowerment, 

• Seamless management tools, and 

• Proactive identification and management of 
risk. 

Organizing for System Development 

Most DoD program offices are part of a Program 
Executive Office (PEO) organization that is usu¬ 
ally supported by a functional organization, such 
as a systems command. Contractors and other gov¬ 
ernment activities provide additional necessary 
support. Establishing a system development orga¬ 
nization requires a network of teams that draw from 
all these organizations. This network, sometimes 
referred to as the enterprise, represents the inter¬ 
ests of all the stakeholders and provides vertical 
and horizontal communications. 

These integrated teams are structured using the 
WBS and designed to provide the maximum 


vertical and horizontal communication during the 
development process. Figure 18-1 shows how team 
structuring is usually done. At the system level 
there is usually a management team and a design 
team. The management team would normally con¬ 
sist of the government and contractor program 
managers, the deputy program manager(s), possi¬ 
bly the contractor Chief Executive Officer, the 
contracting officer, major advisors picked by the 
program manager, the system design team leader, 
and other key members of the system design team. 
The design team usually consists of the first-level 
subsystem and life-cycle integrated team leaders. 

The next level of teams is illustrated on Figure 18-1 
as either product or process teams. These teams 
are responsible for designing system segments 
(product teams) or designing the supporting or 
enabling products (process teams). At this level 
the process teams are coordinating the system level 
process development. For example, the support 
team will integrate the supportability analysis from 
the parts being generated in lower-level design and 


System Level 
Management Team 


System Level 
Design Team 



Sub-Tier Teams 
(Sub-Product or 
Process Oriented) 



Figure 18-1. Integrated Team Structure 
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support process teams. Teams below this level con¬ 
tinue the process at a lower level of decomposi¬ 
tion. Teams are formed only to the lowest level 
necessary to control the integration. DoD team 
structures rarely extend lower than levels three or 
four on the WBS, while contractor teams may ex¬ 
tend to lower levels, depending on the complexi¬ 
ties of the project and the approach favored by 
management. 

The team structure shown by Figure 18-1 is a 
hierarchy that allows continuous vertical commu¬ 
nication. This is achieved primarily by having the 
team leaders, and, if appropriate, other key 
members of a team, be team members of the next 
highest team. In this manner the decisions of the 
higher team is immediately distributed and 
explained to the next team level, and the decisions 
of the lower teams are presented to the higher team 
on a regular basis. Through this method decisions 
of lower-level teams follow the decision making 
of higher teams, and the higher-level teams’ 


decisions incorporate the concerns of lower-level 
teams. 

The normal method to obtain horizontal commu¬ 
nication is shown in Figure 18-2. At least one team 
member from the Product A Team is also a member 
of the Integration and Test Team. This member 
would have a good general knowledge of both 
testing and Product A. The member’s job would 
be to assist the two teams in designing their end or 
enabling products, and in making each understand 
how their decisions would impact the other team. 
Similarly, the member that sits on both Product A 
and B teams would have to understand the both 
technology and the interface issues associated with 
both items. 

The above is an idealized case. Each type of sys¬ 
tem, each type of contractor organization, and each 
level of available resources requires a tailoring of 
this structure. With each phase the focus and the 
tasks change and so should the structure. As phases 



Figure 18-2. Cross Membership 
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are transited, the enterprise structure and team 
membership should be re-evaluated and updated. 

18.2 INTEGRATED TEAMS 

Integrated teams are composed of representatives 
from all appropriate primary functional disciplines 
working together with a team leader to: 

• Design successful and balanced products, 

• Develop the configuration for successful life- 
cycle control, 

• Identify and resolve issues, and 

• Make sound and timely decisions. 

The teams follow the disciplined approach of the 
systems engineering process stalling with require¬ 
ments analysis through to the development of con¬ 
figuration baselines as explained earlier in this 
book. The system-level design team should be 
responsible for systems engineering management 
planning and execution. The system-level manage¬ 
ment team, the highest level program IPT, is 
responsible for acquisition planning, resource 
allocation, and management. Lower-level teams are 
responsible for planning and executing their own 
processes. 

Team Organization 

Good teams do not just happen; they arc the result 
of calculated management decisions and actions. 
Concurrent with development of the enterprise 
organization discussed above, each team must also 
be developed. Basically the following are key 
considerations in planning for a team within an 
enterprise network: 

• The team must have appropriate representation 
from the primary functions, technical special¬ 
ties, and business support, 

• There must be links to establish vertical and 
horizontal communication in the enterprise, 


• You should limit over-uses of cross member¬ 
ship. Limit membership on three or four teams 
as a rough rule of thumb for the working level, 
and 

• Ensure appropriate representation of govern¬ 
ment, contractor, and vendors to assure inte¬ 
gration across key organizations. 

Team Development 

When teams are formed they go through a series 
of phases before a synergistic self-actuating team 
is evolved. These phases are commonly referred 
to as forming, storming, norming and performing. 
The timing and intensity of each phase will depend 
on the team size, membership personality, effec¬ 
tiveness of the team building methods employed, 
and team leadership. The team leaders and an 
enterprise-level facilitator provide leadership 
during the team development. 

Forming is the phase where the members are in¬ 
troduced to their responsibilities and other mem¬ 
bers. During this period members will tend to need 
a structured situation with clarity' of puipose and 
process. If members are directed during this ini¬ 
tial phase, their uncertainty and therefore appre¬ 
hension is reduced. Facilitators controlling the team 
building should give the members rules and tasks, 
but gradually reduce the level of direction as the 
team members begin to relate to each other. As 
members become more familial - with other mem¬ 
bers, the rules, and tasks, they become more com¬ 
fortable in their environment and begin to interact 
at a higher level. 

This starts the storming phase. Storming is the con¬ 
flict brought about by interaction relating to the 
individuals’ manner of dealing with the team tasks 
and personalities. Its outcome is members who 
understand the way they have to act with other 
members to accomplish team objectives. The dy¬ 
namics of storming can be very complex and in¬ 
tense, making it the critical phase. Some teams will 
go through it quickly without a visible ripple, oth¬ 
ers will be loud and hot, and some will never 
emerge from this phase. The team building facili¬ 
tators must be alert to dysfunctional activity. 
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Members may need to be removed or teams 
reorganized. Facilitators during this period must 
act as coaches, directing but in a personal collabo¬ 
rative way. They should also be alert for members 
that are avoiding storming, because the team will 
not mature if there are members who are not 
personally committed to participate in it. 

Once the team has learned to interact effectively it 
begins to shape its own processes and become more 
effective in joint tasks. It is not unusual to see some 
reoccurrence of storming, but if the storming phase 
was properly transitioned these incidences should 
be minor and easily passed. In this phase, norming, 
the team building facilitators become a facilitator 
to the team—not directing, but asking penetrating 
questions to focus the members. They also monitor 
the teams and correct emerging problems. 

As the team continues to work together on their 
focused tasks, their performance improves until 
they reach a level of self-actuation and quality 
decision making. This phase, performing, can take 
a while to reach, 18 months to two years for a 
system-level design team would not be uncommon. 
During the performing stage, the team building 
facilitator monitors the teams and corrects 
emerging problems. 

At the start of a project or program effort, team 
building is commonly done on an enterprise basis 
with all teams brought together in a team-building 
exercise. There are two general approaches to the 
exercise: 

• A team-learning process where individuals are 
given short but focused tasks that emphasize 
group decision, trust, and the advantages of 
diversity. 

• A group work-related task that is important but 
achievable, such as a group determination of 
the enterprise processes, including identifying 
and removing non-value added traditional 
processes. 

Usually these exercises allow the enterprise to 
pass through most of the storming phase if done 


correctly. Three weeks to a month is reasonable 
for this process, if the members are in the same 
location. Proximity does matter and the team build¬ 
ing and later team performance are typically better 
if the teams are co-located. 


18.3 TEAM MAINTENANCE 

Teams can be extremely effective, but they can be 
fragile. The maintenance of the team structure is 
related to empowerment, team membership issues, 
and leadership. 

Empowerment 

The term empowerment relates to how responsi¬ 
bilities and authority is distributed throughout the 
enterprise. Maintenance of empowerment is 
important to promote member ownership of the 
development process. If members do not have 
personal ownership of the process, the effective¬ 
ness of the team approach is reduced or even 
neutralized. The quickest way to destroy partici¬ 
pant ownership is to direct, or even worse, over¬ 
turn solutions that are properly the responsibility 
of the team. The team begins to see that the 
responsibility for decisions is at a higher level 
rather than at their level, and their responsibility is 
to follow orders, not solve problems. 

Empowerment requires: 

• The flow of authority through the hierarchy of 
teams, not through personal direction (irrespec¬ 
tive of organizational position). Teams should 
have clear tasking and boundaries established 
by the higher-level teams. 

• Responsibility for decision making to be 
appropriate for the level of team activity. This 
requires management and higher-level teams to 
be specific, clear, complete, and comprehensive 
in establishing focus and tasking, and in speci¬ 
fying what decisions must be coordinated with 
higher levels. They should then avoid imposing 
or overturning decisions more properly in the 
realm of a lower level. 
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• Teams at each level be given a clear understand¬ 
ing of their duties and constraints. Within the 
bounds of those constraints and assigned duties 
members should have autonomy. Higher-level 
teams and management either accept their 
decisions, or renegotiate the understanding of 
the task. 

Membership Issues 

Another maintenance item of import is team mem¬ 
ber turnover. Rotation of members is a fact of life, 
and a necessary process to avoid teams becoming 
too closed. However, if the team has too fast a turn¬ 
over, or new members are not fully assimilated, 
the team performance level will decline and possi¬ 
bly revert to storming. The induction process 
should be a team responsibility that includes the 
immediate use of the new team member in a jointly 
performed, short term, easily achievable, but 
important task. 

Teams are responsible for their own performance, 
and therefore should have significant, say over the 
choice of new members. In addition teams should 
have the power to remove a member; however, this 
should be preceded by identification of the prob¬ 
lem and active intervention by the facilitator. 
Removal should be a last resort. 

Awards for performance should, where possible, 
be given to the team rather than individuals (or 
equally to all individuals on the team). This 
achieves several things: it establishes a team focus, 
shows recognition of the team as a cohesive force, 
recognizes that the quality of individual effort is 
at least in paid due to team influence, reinforces 
the membership’s dedication to team objectives, 
and avoids team member segregation due to uneven 
awards. Some variation on this theme is appropri¬ 
ate where different members belong to different 
organizations, and a common award system does 
not exist. The system-level management team 
should address this issue, and where possible assure 
equitable awards are given team members. A very 
real constraint on cash awards in DoD rises in the 
case of teams that include both civilian and mili¬ 
tary members. Military members cannot be given 


cash awards, while civilians can. Con-sequently, 
managers must actively seek ways to reward all 
team members appropriately, leaving no group out 
at the expense of others. 

Leadership 

Leadership is provided primarily by the organiza¬ 
tional authority responsible for the program, the 
enterprise facilitator, and the team leaders. In a 
DoD program, the organizational leaders are usu¬ 
ally the program manager and contractor senior 
manager. These leaders set the tone of the enter¬ 
prise adherence to empowerment, the focus of the 
technical effort, and the team leadership of the 
system management team. These leaders are 
responsible to see that the team environment is 
maintained. They should coordinate their action 
closely with the facilitator. 

Facilitators 

Enterprises that have at least one facilitator find 
that team and enterprise performance is easier to 
maintain. The facilitator guides the enterprise 
through the team building process, monitors the 
team network through metrics and other feed¬ 
back, and makes necessary corrections through 
facilitation. The facilitator position can be: 

• A separate position in the contractor organiza¬ 
tion, 

• Part of the responsibilities of the government 
systems engineer or contractor project manager, 
or 

• Any responsible position in the first level below 
the above that is related to risk management. 

Obviously the most effective position would be one 
that allows the facilitator to concentrate on the 
teams’ performance. Enterprise level facilitators 
should have advanced facilitator training and 
(recommended) at least a year of mentored expe¬ 
rience. Facilitators should also have significant 
broad experience in the technical area related to 
the development. 
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Team Leaders 

The team leaders are essential for providing and 
guiding the team focus, providing vertical com¬ 
munication to the next level, and monitoring the 
team’s performance. Team leaders must have a 
clear picture of what constitutes good performance 
for their team. They are not supervisors, though in 
some organizations they may have supervisory 
administrative duties. The leader’s primary puipose 
is to assure that the environment is present that 
allows the team to perform at its optimum level— 
not to direct or supervise. 

The team leader’s role includes several difficult 
responsibilities: 

• Taking on the role of coach as the team forms, 

• Facilitating as the team becomes self-sustaining, 

• Sometimes serving as director (only when a 
team has failed, needs refocus or correction, and 
is done with the facilitator), 

• Providing education and training for members, 

• Facilitating team learning, 

• Representing the team to upper management 
and the next higher-level team, and 

• Facilitating team disputes. 

Team leaders should be trained in basic facilitator 
principles. This training can be done in about a 
week, and there are numerous training facilities or 
companies that can offer it. 

18.4 TEAM PROCESSES 

Teams develop their processes from the principles 
of system engineering management as presented 
earlier in the book. The output of the teams is 
the design documentation associated with prod¬ 
ucts identified on the system architecture, includ¬ 
ing both end product components and enabling 
products. 


Teams use several tools to enhance their pro¬ 
ductivity and improve communication among 
enterprise members. Some examples are: 

• Constructive modeling (CAD/CAE/CAM/ 
CASE) to enhance design understanding and 
control, 

• Trade-off studies and prioritization, 

• Event-driven schedules, 

• Prototyping, 

• Metrics, and most of all 

• Integrated membership that represents the life 
cycle stakeholders. 

Integrated Team Rules 

The following is a set of general rules that should 
guide the activities and priorities of teams in a 
system design environment: 

• Design results must be communicated clearly, 
effectively, and timely. 

• Design results must be compatible with initially 
defined requirements. 

• Continuous “up-the-line” communication must 
be institutionalized. 

• Each member needs to be familiar with all 
system requirements. 

• Everyone involved in the team must work from 
the same database. 

• Only one member of the team has the authority 
to make changes to one set of master documen¬ 
tation. 

• All members have the same level of authority 
(one person, one vote). 

• Team participation is consistent, success- 
oriented, and proactive. 
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• Team discussions are open with no secrets. 

• Team member disagreements must be reasoned 
disagreement (alternative plan of action versus 
unyielding opposition). 

• Trade studies and other analysis techniques are 
used to resolve issues. 

• Issues are raised and resolved early. 

• Complaints about the team are not voiced 
outside the team. Conflicts must be resolved 
internally. 

Guidelines for Meeting Management 

Even if a team is co-located as a work unit, regular 
meetings will be necessary. These meetings and 
their proper running become even more important 
if the team is not co-located and the meeting is the 
primary means of one-on-one contact. A well-run 
technical meeting should incorporate the following 
considerations: 

• Meetings should be held only for a specific 
purpose and a projected duration should be 
targeted. 

• Advance notice of meetings should normally 
be at least two weeks to allow preparation and 
communication between members. 

• Agendas, including time allocations for topics 
and supportive material should be distributed 
no less than three business days before the team 
meeting. The objective of the meeting should 
be clearly defined. 

• Stick to the agenda during the meeting. Then 
cover new business. Then review action items. 

• Meeting summaries should record attendance, 
document any decision or agreements reached, 
document action items and associated due- 
dates, provide a draft agenda for the next 
meeting, and frame issues for higher-level 
resolution. 


• Draft meeting summaries should be provided 
to members within one working day of the 
meeting. A final summary should be issued 
within two working days after the draft 
comments deadline. 


18.5 BARRIERS TO INTEGRATION 

There are numerous barriers to building and main¬ 
taining a well functioning team organization, and 
they are difficult to overcome. Any one of these 
banders can negate the effectiveness of an inte¬ 
grated development approach. Common barriers 
include: 

• Lack of top management support, 

• Team members not empowered, 

• Lack of access to a common database, 

• Lack of commitment to a cultural change, 

• Lunctional organization not fully integrated into 
a team process, 

• Lack of planning for team effort, 

• Staffing requirements conflict with teams, 

• Team members not collocated, 

• Insufficient team education and Paining, 

• Lessons learned and successful practices not 
shared across teams, 

• Inequality of team members, 

• Lack of commitment based on perceived 
uncertainty, 

• Inadequate resources, and 

• Lack of required expertise on either the part of 
the contractor or government. 
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Breaking Barriers 

Common methods to combat barriers include: 

• Education and training, and then more educa¬ 
tion and training: it breaks down the uncertainty 
of change, and provides a vision and method 
for success. 

• Use a facilitator not only to build and maintain 
teams, but also to observe and advise manage¬ 
ment. 

• Obtain management support up front. Manage¬ 
ment must show leadership by managing the 
teams’ environment rather than trying to manage 
people. 

• Use a common database open to all enterprise 
members. 

• Establish a network of teams that integrates the 
design and provides horizontal and vertical 
communication. 

• Establish a network that does not over-tax avail¬ 
able resources. Where a competence is not avail¬ 
able in the associated organizations, hire it 
through a support contractor. 


• Where co-location is not possible have regular 
working sessions of several days duration. Tele¬ 
communications, video conferencing, and other 
technology based techniques can also go far to 
alleviate the problems of non-collocation. 

Summary Comments 

• Integrating system development is a systems 
engineering approach that integrates all 
essential primary function activities through the 
use of multi-disciplinary teams, to optimize the 
design, manufacturing and supportability 
processes. 

• Team building goes through four phases: 
forming, storming, norming, and performing. 

• Key leadership positions in a program network 
of teams are the program manager, facilitator, 
and team leaders. 

• A team organization is difficult to build and 
maintain. It requires management attention and 
commitment over the duration of the teams 
involved. 
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SUPPLEMENT 18-A 

IPPD - A DOD 
MANAGEMENT PROCESS 


The DoD policy of Integrated Product and Process 
Development (IPPD) is a broad view of integrated 
system development which includes not only 
systems engineering, but other areas involved in 
formal decision making related to system devel¬ 
opment. DoD policy emphasizes integrated 
management at and above the Program Manager 
(PM) level. It requires IPPD at the systems 
engineering level, but does not direct specific 
organizational structures or procedures in recog¬ 
nition of the need to design a tailored IPPD process 
to every individual situation. 

Integrated Product Teams 

One of the key IPPD tenets is multi-disciplinary 
integration and teamwork achieved through the use 
of Integrated Product Teams (IPTs). While IPTs 
may not be the best solution for every manage¬ 
ment situation, the requirement to produce inte¬ 
grated designs that give consideration to a wide 
array of technical and business concerns leads most 
organizations to conclude that IPTs are the best 
organizational approach to systems management. 
PMs should remember that the participation of a 
contractor or a prospective contractor on a IPT 
should be in accordance with statutory require¬ 
ments, such as procurement integrity rules. The 
service component’s legal advisor must review 
prospective contractor involvement on IPTs. To 
illustrate issues the government-contractor team 
arrangement raises, the text box at the end of this 
section lists nine rules developed for government 
members of the Advanced Amphibious Assault 
Vehicle (AAAV) design IPTs. 

The Secretary of Defense has directed that DoD 
perform oversight and review by using IPTs. 
These IPTs function in a spirit of teamwork with 


participants empowered and authorized, to the 
maximum extent possible, to make commitments 
for the organization or the functional area they 
represent. IPTs are composed of representatives 
from all appropriate functional disciplines work¬ 
ing together to build successful programs and en¬ 
abling decision makers to make the right decisions 
at the right time. 

DoD IPT Structure 

The DoD oversight function is accomplished 
through a hierarchy of teams that include levels of 
management from DoD to the program level. There 
are three basic levels of IPTs: the Overaching IPT 
(OIPT), the Working IPTs (WIPT), and Program 
IPTs with the focus and responsibilities as shown 
by Figure 18-3. For each ACAT I program, there 
will be an OIPT and at least one WIPT. WIPTs 
will be developed for particular functional topics, 
e.g., test, cost/performance, contracting, etc. An 
Integrating IPT (IIPT) will coordinate WIPT efforts 
and cover all topics not otherwise assigned to 
another IPT. These teams are structurally organized 
as shown on Figure 18-4. 

Overarching IPT (OIPT) 

The OIPT is a DoD level team whose primary re¬ 
sponsibility is to advise the Defense Acquisition 
Executive on issues related to programs managed 
at that level. The OIPT membership is made up of 
the principals that are charged with responsibility 
for the many functional offices at the Office of the 
Secretary of Defense (OSD). 

The OIPT provides: 

• Top-level strategic guidance, 
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Organization 

Teams 

Focus 

Participant 

Responsibilities 

OSD and 
Components 

01PT* 

• Strategic Guidance 

• Tailoring 

• Program Assessment 

• Resolve Issues Elevated by WIPTs 

• Program Success 

• Functional Area Leadership 

• Independent Assessment 

• Issue Resolution 


WIPTs* 

• Planning for Program Success 

• Opportunities for Acquisition 

Reform (e.g. innovation, streamlining) 

• Identify/Resolve Program Issues 

• Program Status 

• Functional Knowledge and Experience 

• Empowered Contribution 

• Recom.’s for Program Success 

• Communicate Status and Unresolved 
Issues 

Program 

Teams and 

System 

Contractors 

Program 

IPTs** 

• Program Execution 

• Identify and Implement Acquisition 
Reform 

• Manage Complete Scope of Program 
Resources, and Risk 

• Integrate Government and Contractor 
Efforts for Report Program Status and 
Issues 

* Covered in “Rules of the Road” 

** Covered in “Guide to Implementation and Management of IPPD in DoD Acquisition” 


Figure 18-3. Focus and Responsibilities of IPTs 
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• Functional area leadership, 

• Forum for issue resolution, 

• Independent assessment to the MDA, 

• Determine decision information for next 
milestone review, and 

• Provide approval of the WIPT structures and 
resources. 

Working-Level IPT (WIPT) 

The WIPTs may be thought of as teams that link 
the PM to the OIPT. WIPTs are typically func¬ 
tionally specialized teams (test, cost-performance, 
etc.). The PM is the designated head of the WIPT, 
and membership typically includes representation 
from various levels from the program to OSD staff. 
The principal functions of the WIPT are to advise 
the PM is the area of specialization and to advise 
the OIPT of program status. 

The duties of the WIPT include: 

• Assisting the PM in developing strategies and 
in program planning, as requested by the PM, 

• Establishing IPT plan of action and milestones, 


• Proposing tailored document and milestone 
requirements, 

• Reviewing and providing early input to docu¬ 
ments, 

• Coordinating WIPT activities with the OIPT 
members, 

• Resolving or evaluating issues in a timely 
manner, and 

• Obtaining principals’ concurrence with appli¬ 
cable documents or portions of documents. 

Program IPTs 

Program IPTs are teams that perform the program 
tasks. The integration of contractors with the gov¬ 
ernment on issues relative to a given program truly 
occurs at the program IPT level. The development 
teams (product and process teams) described ear¬ 
lier in this chapter would be considered program 
IPTs. Program IPTs would also include teams 
formed for business reasons, for example teams 
established to prepare Planning, Programming, and 
Budgeting System (PPBS) documentation, to pre¬ 
pare for Milestone Approval, to develop the RFP, 
or the like. 
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SUPPLEMENT 18-B 

GOVERNMENT ROLE ON IPTs 


The following list was developed by the Advanced 
Amphibious Assault Vehicle (AAAV) program to in- 
form its government personnel of their role on con- 
tractor/government integrated teams. It addresses 
government responsibilities and the realities im¬ 
posed by contractual and legal constraints. Though 
it is specific to the AAAV case, it can be used as 
guidance in the development of team planning for 
other programs. 

1. The IPTs are contractor-run entities. We do not 
lead or manage the IPTs. 

2. We serve as “customer” representatives on the 
IPTs. We are there to REDUCE THE CYCLE 
TIME of Contractor-Government (customer) 
communication. In other words, we facilitate 
contractor personnel getting Government 
input faster. Government IPT members also 
enable us to provide the contractor IPT Status 
and issue information up the Government 
chain on a daily basis (instead of monthly or 
quarterly). 

3. WE DO NOT DO the contractor’s IPT WORK, 
or any portion of their work or tasks. The con¬ 
tractor has been contracted to perform the tasks 
outlined in the contract SOW; their personnel 
and their subcontractors’ personnel will per¬ 
form those tasks, not us. But Government IPT 
members will be an active paid of the delib¬ 
erations during the development of, and par¬ 
ticipate in “on-the-fly” reviews of deliverables 
called out in CDRLs. 

4. When asked by contractor personnel for the 
Government’s position or interpretation, Gov¬ 
ernment IPT members can offer their personal 
opinion, as an IPT member, or offer expert 
opinion; you can provide guidance as to our 


“customer” opinion and what might be 
acceptable to the Government but you can only 
offer the “Government” position for items that 
have been agreed to by you and your Supervi¬ 
sor. IT IS UP TO YOUR SUPERVISORS TO 
EMPOWER EACH OF YOU TO AN APPRO¬ 
PRIATE LEVEL OF AUTHORITY. It is ex¬ 
pected that this will start at a minimal level of 
authority and be expanded as each individual’s 
IPT experience and program knowledge 
grows. However... (see items 5 and 6). 

5. Government IPT members CAN NOT autho¬ 
rize any changes or deviations to/from the con¬ 
tract SOW or Specifications. Government IPT 
members can participate in the deliberations 
and discussions that would result in the sug¬ 
gestion of such changes. IfAVhen an IPT con¬ 
cludes that the best course of action is not in 
accordance with the contract, and a contract 
change is in order, then the contractor must 
submit a Contract Change Request (CCR) 
through normal channels. 

6. Government IPT members CAN NOT autho¬ 
rize the contractor to perform work that is in 
addition to the SOW/contract requirements. 
The contractor IPTs can perform work that is 
not specifically required by the contract, at 
their discretion (provided they stay within the 
resources as identified in the Team Operating 
Contract (TOC). 

7. Government IPT member participation in 
contractor IPT activities IS NOT Government 
consent that the work is approved by the Gov¬ 
ernment or is chargeable to the contract. If an 
IPT is doing something questionable, identify 
it to your supervisor or Program Management 
Team (PMT) member. 
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8. Government members of IPTs do not approve 
or disapprove of IPT decisions, plans, or 
reports. You offer your opinion in their 
development, you vote as a member, and you 
coordinate issues with your Supervisor and 
bring the “Government” opinion (in the form 
of your opinion) back to the IPT, with the goal 
of improving the quality of the products; you 
don’t have veto power. 

9. Government IPT members are still subject to 
all the Government laws and regulations re¬ 
garding “directed changes,” ethics, and con¬ 
duct. Your primary function is to perform those 
functions that are best done by Government 
employees, such as: 


• Conveying to contractor personnel your 
knowledge/expertise on Marine Corps 
operations and maintenance techniques; 

• Interfacing with all other Government 
organizations (e.g., T&E); 

• Control/facilitization of government fur¬ 
nished equipment and materials (GFE and 
GFM); 

• Ensuring timely payment of submitted 
vouchers; and 

• Full participation in Risk Management. 
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19.1 INTRODUCTION 

This chapter describes how the systems engineer 
supports the development and maintenance of the 
agreement between the project office and the con¬ 
tractor that will perform or manage the detail work 
to achieve the program objectives. This agreement 
has to satisfy several stakeholders and requires 
coordination between responsible technical, mana¬ 
gerial, financial, contractual, and legal personnel. 
It requires a document that conforms to the Fed¬ 
eral Acquisition Regulations (and supplements), 
program PPBS documentation, and the System 
Architecture. As shown by Figure 19-1, it also has 
to result in a viable cooperative environment that 
allows necessary integrated teaming to take place. 


The role of technical managers or systems engi¬ 
neers is crucial to satisfying these diverse concerns. 

Their primary responsibilities include: 

• Supporting or initiating the planning effort. 
The technical risk drives the schedule and cost 
risks which in turn should drive the type of 
contractual approach chosen, 

• Prepares or supports the preparation of the 
source selection plan and solicitation clauses 
concerning proposal requirements and selection 
criteria, 

• Prepares task statements, 



Figure 19-1. Contracting Process 
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• Prepares the Contract Data Requirements List 
(CDRL), 

• Supports negotiation and participates in source 
selection evaluations, 

• Forms Integrated Teams and coordinates the 
government side of combined government and 
industry integrated teams, 

• Monitors the contractor’s progress, and 

• Coordinates government action in support of 
the contracting officer. 

This chapter reflects the DoD approach to contract¬ 
ing for system development. It assumes that there 
is a government program or project office that is 
tasking a prime contractor in a competitive envi¬ 
ronment. Flowever, in DoD there is variation to 
this theme. Some project activities are tasked di¬ 
rectly to a government agency or facility, or are 
contracted sole source. The processes described 
in this chapter should be tailored as appropriate 
for these situations. 


19.2 SOLICITATION DEVELOPMENT 

As shown by Figure 19-2, the DoD contracting 
process begins with planning efforts. Planning in¬ 
cludes development of a Request for Proposal 
(RFP), specifications, a Statement of Objective 
(SOO) or Statement of Work (SOW), a source 
selection plan, and the Contract Data Requirements 
List (CDRL). 

Request for Proposal (RFP) 

The RFP is the solicitation for proposals. The gov¬ 
ernment distributes it to potential contractors. It 
describes the government’s need and what the 
offeror must do to be considered for the contract. 
It establishes the basis for the contract to follow. 

The key systems engineering documents included 
in a solicitation are: 

• A statement of the work to be performed. In 
DoD this is a SOW. A SOO can be used to ob¬ 
tain a SOW or equivalent during the selection 
process. 


Acquisition Planning 



Figure 19-2. Contracting Process 
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• A definition of the system. Appropriate speci¬ 
fications and any additional baseline informa¬ 
tion necessary for clarification form this 
documentation. This is generated by the systems 
engineering process as explained earlier in this 
book. 

• A definition of all data required by the customer. 
In DoD this accomplished through use of the 
Contract Data Requirements List (CDRL). 

The information required to be in the proposals 
responding to the solicitation is also key for the 
systems engineer. An engineering team will decide 
the technical and technical management merits of 
the proposals. If the directions to the offerors are 
not clearly and correctly stated, the proposal will 
not contain the information needed to evaluate the 
offerors. In DoD Sections L and M of the RFP are 
those pivotal documents. 

Task Statement 

The task statement prepared for the solicitation will 
govern what is actually received by the govern¬ 
ment, and establish criteria forjudging contractor 
performance. Task requirements arc expressed in 


the SOW. During the solicitation phase the tasks 
can be defined in very general way by a SOO. 
Specific details concerning SOOs and SOWs are 
attached at the end of this chapter. 

As shown by Figure 19-3, solicitation tasking 
approaches can be categorized into four basic op¬ 
tions: use of a basic operational need, a SOO, a 
SOW, or a detail specification. 

Option 1 maximizes contractor flexibility by sub¬ 
mitting the Operational Requirements Document 
(ORD) to offerors as a requirements document (e.g. 
in place of SOO/SOW), and the offerors are re¬ 
quested to propose a method of developing a 
solution to the ORD. The government identifies 
its areas of concern in Section M (evaluation fac¬ 
tors) of the RFP to provide guidance. Section L 
(instructions to the offerors) should require the 
bidders write a SOW based on the ORD as part of 
their proposal. The offeror proposes the type of 
system. The contractor develops the system speci¬ 
fication and the Work Breakdown Structure 
(WBS). In general this option is appropriate for 
early efforts where contractor input is necessary 
to expand the understanding of physical solutions 
and alternative system approaches. 


Government Develops 


Contractor Develops 
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Select — 
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Select— 
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Figure 19-3. Optional Approaches 
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Option 2 provides moderate contractor flexibility 
by submitting a SOO to the offerors as the Section 
C task document (e.g., in place of SOW.) The gov¬ 
ernment identifies its areas of concern in Section 
M (evaluation factors) to provide guidance. Sec¬ 
tion L (instructions to the offerors) should require 
as part of the proposal that offerors write a SOW 
based on the SOO. In this case the government 
usually selects the type of system, writes a draft 
technical-requirements document or system speci¬ 
fication, and writes a draft WBS. This option is 
most appropriate when previous efforts have not 
defined the system tightly. The effort should not 
have any significant design input from the previ¬ 
ous phase. This method allows for innovative think¬ 
ing by the bidders in the proposal stage. It is a 
preferred method for design contracts. 

Option 3 lowers contractor flexibility, and in¬ 
creases clarity of contract requirements. In this 
option the SOW is provided to the Contractor as 
the contractual task requirements document. The 
government provides instructions in Section L to 
the offerors to describe the information needed by 
the government to evaluate the contractor’s ability 
to accomplish the SOW tasks. The government 
identifies evaluation factors in Section M to pro¬ 
vide guidance for priority of the solicitation re¬ 
quirements. In most cases, the government selects 
the type of system, and provides the draft system 
spec, as well as the draft WBS. This option is most 
appropriate when previous efforts have defined the 
system to the lower WBS levels or where the 
product baseline defines the system. Specifically 
when there is substantial input from the previous 
design phase and there is a potential for a different 
contractor on the new task, the SOW method is 
appropriate. 

Option 4 minimizes contractor flexibility, and 
requires maximum clarity and specificity of con¬ 
tract requirements. This option uses an Invitation 
for Bid (IFB) rather than an RFP. It provides bid¬ 
ders with specific detailed specifications or task 
statements describing the contract deliverables. 
They tell the contractor exactly what is required 
and how to do it. Because there is no flexibility in 
the contractual task, the contract is awarded based 
on the low bid. This option is appropriate when 


the government has detailed specifications or 
other product baseline documentation that de¬ 
fines the deliverable item sufficient for produc¬ 
tion. It is generally used for simple build-to-print 
reprocurement. 

Data Requirements 

As part of the development of an IFB or RFP, the 
program office typically issues a letter that de¬ 
scribes the planned procurement and asks inte¬ 
grated team leaders and affected functional man¬ 
agers to identify and justify their data requirements 
for that contract. The data should be directly as¬ 
sociated with a process or task the contractor is 
required to perform. 

The affected teams or functional offices then 
develop a description of each data item needed. 
Data Item Descriptions (DIDs), located in the 
Acquisition Management Systems and Data 
Requirements Control List (AMSDL), can be used 
for guidance in developing these descriptions. 
Descriptions should be performance based, and 
format should be left to the contractor as long as 
all pertinent data is included. The descriptions are 
then assembled and submitted for inclusion in the 
solicitation. The listing of data requirements in the 
contract follows an explicit format and is referred 
to as the CDRL. 

In some cases the government will relegate the data 
call to the contractor. In this case it is important 
that the data call be managed by a government/ 
contractor team, and any disagreements be resolved 
prior to formal contract change incorporating data 
requirements. When a SOO approach is used, the 
contractor should be required by section L to pro¬ 
pose data requirements that correspond to their 
proposed SOW. 

There is current emphasis on electronic submis¬ 
sion of contractually required data. Electronic Data 
Interchange (EDI) sets the standards for compatible 
data communication formats. 

Additional information on data management, 
types of data, contractual considerations, and 
sources of data are presented in Chapters 10 and 
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13. Additional information on CDRLs is provided 
at the end of this chapter. 

Technical Data Package Controversy 

Maintenance of a detailed baseline such as the “as 
built” description of the system, usually referred 
to as a Technical Data Package (TDP), can be very 
expensive and labor intensive. Because of this, 
some acquisition programs may not elect to pur¬ 
chase this product description. If the Government 
will not own the TDP the following questions must 
be resolved prior to solicitation issue: 

• What are the pros and cons associated with the 
TDP owned by the contractor? 

• What arc the support and reprocurement impacts? 

• What are the product improvement impacts? 

• What are the open system impacts? 

In general the government should have sufficient 
data rights to address life cycle concerns, such as 
maintenance and product upgrade. The extent to 
which government control of configurations and 
data is necessary will depend on support and 
reprocurement strategies. This, in turn, demands 
that those strategic decisions be made as early as 
possible in the system development to avoid pur¬ 
chasing data rights as a hedge against the possibility 
that the data will be required later in the program 
life cycle. 

Source Selection 

Source Selection determines which offeror will be 
the contractor, so this choice can have profound 
impact on program risk. The systems engineer must 
approach the source selection with great care 
because, unlike many planning decisions made 
early in product life cycles, the decisions made 
relative to source selection can generally not be 
easily changed once the process begins. Laws and 
regulations governing the fairness of the process 
require that changes be made very carefully—and 
often at the expense of considerable time and effort 
on the part of program office and contractor 


personnel. In this environment, even minor 
mistakes can cause distortion of proper selection. 

The process starts with the development of a 
Source Selection Plan (SSP), that relates the orga¬ 
nizational and management structure, the evalua¬ 
tion factors, and the method of analyzing the 
offerors’ responses. The evaluation factors and their 
priority are transformed into information provided 
to the offerors in sections L and M of the RFP The 
offerors’ proposals are then evaluated with the pro¬ 
cedures delineated in the SSP. These evaluations 
establish which offerors are conforming, guide 
negotiations, and are the major factor in contrac¬ 
tor selection. The SSP is further described at the 
end of this chapter. 

The system engineering area of responsibility 
includes support of SSP development by: 

• Preparing the technical and technical manage¬ 
ment parts of evaluation factors, 

• Organizing technical evaluation team(s), and 

• Developing methods to evaluate offerors’ pro¬ 
posals (technical and technical management). 

19.3 SUMMARY COMMENTS 

• Solicitation process planning includes develop¬ 
ment of a Request for Proposal, specifications, 
a Statement of Objective or Statement of Work, 
a source selection plan, and the Contract Data 
Requirements List. 

• There are various options available to program 
offices as far as the guidance and constraints 
imposed on contractor flexibility. The govern¬ 
ment, in general, prefers that solicitations be 
performance-based. 

• Data the contractor is required to provide the 
government is listed on the CDRL List. 

• Source Selection is based on the evaluation 
criteria outlined in the SSP and reflected in 
Sections L and M of the RFP. 
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SUPPLEMENT 19-A 

STATEMENT OF OBJECTIVES 

(SOO) 


The SOO is an alternative to a government pre¬ 
pared SOW. A SOO provides the Government’s 
overall objectives and the offeror’s required sup¬ 
port to achieve the contractual objectives. Offerors 
use the SOO as a basis for preparing a SOW which 
is then included as an integral part of the proposal 
which the government evaluates during the source 
selection. 

Purpose 

SOO expresses the basic, top-level objectives of 
the acquisition and is provided in the RFP in lieu 
of a government-written SOW. This approach gives 
the offerors the flexibility to develop cost effec¬ 
tive solutions and the opportunity to propose 
innovative alternatives. 

Approach 

The government includes a brief (1- to 2-page) 
SOO in the RFP and requests that offerors provide 
a SOW in their proposal. The SOO is typically 
appended to section J of the RFP and does not be¬ 
come part of the contract. Instructions for the con¬ 
tractor prepared SOW would normally be included 
in or referenced by Section L. 

SOO Development 

Step 1: The RFP team develops a set of objectives 
compatible with the overall program direction 
including the following: 

• User(s) operational requirements, 

• Programmatic direction, 

• Draft technical requirements, and 


• Draft WBS and dictionary. 

Step 2: Once the program objectives are defined, 
the SOO is constructed so that it addresses prod¬ 
uct-oriented goals and performance-oriented 
requirements. 

SOO and Proposal Evaluations 

Section L (Instructions to Offerors) of the RFP 
must include instructions to the offeror that require 
using the SOO to construct and submit a SOW. In 
Section M (Evaluation Criteria) the program office 
should include the criteria by which the proposals, 
including the contractor’s draft SOW, will be evalu¬ 
ated. Because of its importance, the government’s 
intention to evaluate the proposed SOW should be 
stressed in Sections L and M. 

Offeror Development of 
the Statement of Work 

The offeror should establish and define in clear, 
understandable terms: 

• Non-specification requirements (the tasks that 
the contractor must do), 

• What has to be delivered or provided in order 
for him to get paid, 

• What data is necessary to support the effort, 
and 

• Information that would show how the offerors 
would perform the work that could differenti¬ 
ate between them in proposal evaluation and 
contractor selection. 
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SOO Example: 

Joint Air-to-Surface Standoff Missile (JASSM) 

Statement of Objectives 

The Air Force and Navy warfighters need a standoff missile that will destroy the enemies’ war- 

sustaining capabilities with a launch standoff range outside the range of enemy area defenses. 

Offerors shall use the following objectives for the pre-EMD and EMD acquisition phases of the 

JASSM program along with other applicable portions of the RFP when preparing proposals and 

program plans. IMP events shall be traceable to this statement of objectives: 

Pre-EMD Objectives 

a. Demonstrate, at the sub-system level as a minimum, end-to-end performance of the sys¬ 
tem concept. Performance will be at the contractor-developed System Performance Speci¬ 
fication requirements level determined during this phase without violation of any key 
performance parameters. 

b. Demonstrate the ability to deliver an affordable and producible system at or under the average 
unit procurement price (AUPP). 

c. Provide a JASSM system review including final system design, technical accomplishments, 
remaining technical risks and major tasks to be accomplished in EMD. 

EMD Objectives 

a. Demonstrate through test and/or analysis that all requirements as stated in the contractor 
generated System Performance Specification, derived from Operational Requirements, are 
met, including military utility (operational effectiveness and suitability). 

b. Demonstrate ability to deliver an affordable and producible system at or under the AUPP 
requirement. 

c. Demonstrate all production processes. 

d. Produce production representative systems for operational test and evaluation, including 
combined development/operational test and evaluation. 


At contract award the SOW, as changed through 
negotiations, becomes part of the contract and the 
standard for measuring contractor’s effectiveness. 
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SUPPLEMENT 19-B 

STATEMENT OF WORK 
(SOW) 


The SOW is a specific statement of the work to be 
performed by the contractor. It is derived from the 
Program WBS (System Architecture). It should 
contain, at a minimum, a statement of scope and 
intent, as well as a logical and clear definition of 
all tasks required. The SOW normally consists of 
three parts: 

Section 1: Scope - Defines overall purpose of the 
program and to what the SOW applies. 

Section 2: Applicable Documents - Lists the 
specifications and standards referenced in Section 
3. 


Section 3: Requirements - States the tasks the 
contractor has to perform to provide the 
deliverables. Tasks should track with the WBS. The 
SOW describes tasks the contractor has to do. The 
specifications describe the products. 

Statement of Work Preparation 
and Evaluation Strategies 

SOWs should be written by an integrated team of 
competent and experienced members. The team 
should: 

• Review and use the appropriate WBS for the 
SOW framework. 



Figure 19-4. Requirement-WBS-SOW Flow 
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• Set SOW objectives in accordance with the 
Acquisition Plan and systems engineering 
planning, 

• Develop a SOW tasking outline and check list, 

• Establish schedule and deadlines, and 

• Develop a comprehensive SOW from the above. 

Performance-based SOW 

The term performance-based SOW has become a 
common expression that relates to a SOW that tasks 
the contractor to perform the duties necessary to 
provide the required deliverables, but is not specific 
as to the process details. Basically, all SOWs should 
be performance based, however, past DoD gener¬ 
ated SOWs have had the reputation of being overly 
directive. A properly developed SOW tasks the 
contractor without telling him how to accomplish 
the task. 

Evaluating the SOW 

The WBS facilitates a logical arrangement of the 
elements of the SOW and a tracing of work effort 
expended under each of the WBS elements. It helps 
integrated teams to ensure all requirements have 
been included, and provides a foundation for hack¬ 
ing program evolution and controlling the change 
process. As shown by Figure 19-4, the WBS serves 
as a link between the requirements and the SOW. 

In the past, DoD usually wrote the SOW and, over 
time, an informal set of rules had been developed 
to assist in drafting them. While the government 
today generally does not write the SOW, but, rather, 
more often evaluates the contractor’s proposed SOW, 
those same rules can assist in the government role 
of evaluator. 

Statement of Work Rules 

In section 1. Scope: 

DO NOT: 

• Include directed work statements. 


• Include data requirements or deliverable 
products. 

In section 2. Applicable Documents: 

DO NOT: 

• Include guidance documents that apply only to 
Government PMOs (e.g., DoD 5000 series and 
service regulations). 

In section 3. Requirements: 

DO NOT: 

• Define work tasks in terms of data to be deliv¬ 
ered. 

• Order, describe, or discuss CDRL data (OK to 
reference). 

• Express work tasks in data terms. 

• Invoke, cite, or discuss a DID. 

• Invoke handbooks, service regulations, techni¬ 
cal orders, or any other document not specifi¬ 
cally written in accordance with MIL-STD-961/ 
962. 

• Specify how task is to be accomplished. 

• Use the SOW to amend contract specifications. 

• Specify technical proposal or performance 
criteria or evaluation factors. 

• Establish delivery schedules. 

• Over specify. 

In section 3. Requirements: 

DO: 

• Specify work requirements to be performed 
under contract. 
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• Set SOW objectives to reflect the acquisition • Use WBS as an outline, 
plan and systems engineering planning. 

• List tasks in chronological order. 

• Provide a priceable set of tasks. 

• Limit paragraph numbering to 3rd sub-level 

• Express work to be accomplished in work (3.3.1.1.) - Protect Government interests, 
words. 

• Allow for contractor’s creative effort. 

• Use “shall” whenever a task is mandatory. 

• Use “will” only to express a declaration of 
puipose or simple futurity. 
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SUPPLEMENT 19-C 

CONTRACT DATA 
REQUIREMENTS LIST 

The Contract Data Requirements List (CDRL) is Data requirements can also be identified in the 

a list of authorized data requirements for a specific contract via Special Contract Clauses (Federal 

procurement that forms a part of the contract. It is Acquisition Clauses.) Data required by the FAR 

comprised of a series of DD Forms 1423 (Indi- clauses are usually required and managed by the 

vidual CDRL forms) containing data requirements Contracting Officer. 

and delivery instructions. CDRLs should be linked 

directly to SOW tasks and managed by the program 

office data manager. A sample CDRL data 

requirement is shown in Figure 19-5. 


CONTRACT DATA REQUIREMENTS LIST 


ATCH NR: 3 TO EXHIBIT: 

TO CONTRACT/PR: F33657-86-C-2085 


CATEGORY: X 


SYSTEM/ITEM: ATF DEM/VAL PHASE 
CONTRACTOR: LOCKHEED 


1) 

2) SOW 3.1 

6) 

10) 

12) 

14) 


3100 

3) 

ASD/TASE 

ONE/R 

60DAC 

ASD/TASE 

2/0 


OTE62011 


16) 

BLK4: 


5) SOW 3.1 

7) 

8) 

9) 


IT 

D 




13) 

SEE 16 


ASD/TASM 

ASD/TASL 

ACO 


BLK 4: SEE APPENDIXES TO CDRL FOR DID. 

THIS DID IS TAILORED AS FOLLOWS: 

(1) CONTRACTOR FORMAT IS ACCEPTABLE. 

(2) CHANGE PARAGRAPH 2a OF DID TO READ: “PROGRAM RISK 
ANALYSIS.THIS SECTION SHALL DESCRIBETHE PLAN AND 
METHODOLOGY FOR A CONTINUING ASSESSMENT OF 
TECHNICAL, SUPPORTABILITY, COST, AND SCHEDULE RISKS OF 
THE SYSTEM PROGRAM.THIS SECTION SHOULD BE 
CONSISTENT WITH AND NOT DUPLICATE THE SYSTEM 
INTEGRATION PLAN (REFERENCE DI-S-3563/T); i.e., ONE PLAN 
MAY REFERENCE THE OTHER.” 

BLK 13: REVISIONS SHALL BE SUBMITTED AS REQUIRED BY CHANGE 
RESULTING FROMTHE SYSTEMS ENGINEERING PROCESS. 
NOTE: SCHEDULES ASSOCIATED WITH THIS PLAN SHALL BE 
INTEGRATED WITH THE MASTER PROGRAM PLANNING 
SCHEDULE SUBMITTED ON MAGNETIC MEDIA IN ACCORDANCE 
WITH DI-A-3007/T. 



PREPARED BY: 

DATE: 

APPROVED BY: 

DATE: 


86JUN 11 


86 JUNE 11 


DD FORM 1423 ADPE ADAPTATION SEP 81 (ASD/YYD) 


Figure 19-5. CDRL Single Data Item Requirement Example 
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Data Requirement Sources 

Standard Data Item Descriptions (DID) define data 
content, preparation instructions, format, intended 
use, and recommended distribution of data required 
of the contractor for delivery. The Acquisition 
Management Systems and Data Requirements 
Control List (AMSDL) identifies acquisition man¬ 
agement systems, source documents, and standard 
DIDs. With acquisition reform the use of DIDs has 
declined, and data item requirements now are ei¬ 
ther tailored DIDs or a set of requirements specifi¬ 
cally written for the particular RFP in formats 
agreeable to the contractor and the government. 

DD Form 1423 Road Map 

Block 1: Data Item Number - represents the CDRL 
sequence number. 

Block 2: Title of Data Item - same as the title 
entered in item 1 of the DID (DD Form 1664). 

Block 4: Authority (Data Acquisition Document 
Number) - same as item 2 of the DID form and 
will include a “/t” to indicate DID has been tailored. 

Block 5: Contr act Reference - identifies the DID 
authorized in block 4 and the applicable document 
and paragraph numbers in the SOW from which 
the data flows. 

Block 6: Requiring Office - activity responsible 
for advising the technical adequacy of the data. 

Block 7: Specific Requirements - may be needed 
for inspection/acceptance of data. 

Block 8: Approval Code - if “A,” it is a critical 
data item requiring specific, advanced, written 
approval prior to distr ibution of the final data item. 


Block 9: Distribution Statement Required: 

Category A is unlimited-release to the public. 

Category B is limited-release to government 
agencies. 

Category C limits release to government agencies 
and their contractors. 

Category D is limited-release to DoD offices and 
their contractors. 

Category E is for release to DoD components only. 

Category F is released only as directed and 
normally classified. 

Block 12: Date of First Submission - indicates 
ycar/month/day of first submission and identifies 
specific event or milestone data is required. 

Block 13: Date of Subsequent Submission - if data 
is submitted more than once, subsequent dates will 
be identified. 

Block 14: Distribution - identify each addressee 
and identify the number of copies to be received 
by each. Use office symbols, format of data to be 
delivered, command initials, etc. 

Block 16: Remarks - explain only tailored features 
of the DID, any additional information for blocks 
1-15, and any resubmittal schedule or special con¬ 
ditions for updating data submitted for government 
approval. 
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SUPPLEMENT 19-D 

THE SOURCE 
SELECTION PLAN 


Prior to solicitation issuance, a source selection 
plan should be prepared by the Program Manager 
(PM), reviewed by the Contracting Officer, and 
approved by the Source Selection Authority (SSA). 
A Source Selection Plan (SSP) generally consists 
of three parts: 

• The first part describes the organization, 
membership, and responsibilities of the source 
selection team, 

• The second paid identifies the evaluation factors, 
and 

• The last paid establishes detailed procedures for 
the evaluation of proposals. 

Source Selection Organization 

The SSA is responsible for selecting the source 
whose proposal is most advantageous to the gov¬ 
ernment. The Source Selection Advisory Council 


(SSAC) provides advice to the SSA based on the 
Source Selection Evaluation Board’s (SSEB’s) 
findings and the collective experience of SSAC 
members. The SSEB generates the information the 
SSA needs by performing a comprehensive evalu¬ 
ation of each offeror’s proposal. A Technical Evalu¬ 
ation Review Team(s) evaluates the technical por¬ 
tion of the proposals to support the SSEB. The 
process flow is shown in Figure 19-6. 

The PM is responsible for developing and imple¬ 
menting the acquisition strategy, preparing the SSP, 
and obtaining SSA approval of the plan before the 
formal solicitation is issued to industry. The System 
Engineer or technical manager supports the PM’s 
efforts. The Contracting Officer is responsible for 
preparation of solicitations and contracts, any com¬ 
munications with potential offerors or offerors, 
consistency of the SSP with requirements of the 
Federal Acquisition Regulation (FAR) and DoD 
FAR Supplement (DFARS), and award of the 
contract. 



Figure 19-6. Source Selection Process 
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SSP Evaluation Factors 

The evaluation factors are a list, in order of rela¬ 
tive importance, of those aspects of a proposal that 
will be evaluated quantitatively and qualitatively to 
arrive at an integrated assessment as to which pro¬ 
posal can best meet the Government’s need as 
described in the solicitation. Figure 19-7 shows 
an example of one evaluation category, life cycle 
cost. The purpose of the SSP evaluation is to 
inform offerors of the importance the Govern¬ 
ment attaches to various aspects of a proposal and 
to allow the government to make fair and reasoned 
differentiation between proposals. 

In general the following guidance should be used 
in preparing evaluation factors: 

• Limit the number of evaluation factors, 

• Tailor the evaluation factors to the Government 
requirement (e.g., combined message of the 
SOO/SOW, specification, CDRL, etc.), and 

• Cost is always an evaluation factor. The identi¬ 
fication of the cost that is to be used and its 
relative importance in rating the proposal should 
be clearly identified. 


Factors to Consider 

There is not sufficient space here to attempt to ex¬ 
haustively list all the factors that might influence 
the decision made in a source selection. The 
following are indicative of some of the key 
consideration, however: 

• Is the supplier’s proposal responsive to the 
government’s needs as specified in the RFP? 

• Is the supplier’s proposal directly supportive of 
the system requirements specified in the system 
specification and SOO/SOW? 

• Flave the performance characteristics been 
adequately specified for the items proposed? 
Are they meaningful, measurable , and traceable 
from the system-level requirements? 

• Flave effectiveness factors been specified 
(e.g., reliability, maintainability, supportability, 
and availability?) Are they meaningful, mea¬ 
surable, and traceable, from the system-level 
requirements? 

• Flas the supplier addressed the requirement for 
test and evaluation of the proposed system 
element? 


Rating 

(Points) 

Evaluation Criteria - Life Cycle Cost 

9-10 

Offeror has included a complete Life Cycle Cost analysis that supports their proposal. 

7-8 

Offeror did not include a complete Life Cycle Cost analysis but has supported their 
design approach on the basis of Life Cycle Cost. 

5-6 

Offeror plans to complete a Life Cycle Cost analysis as part of the contract effort and 
has described the process that will be used. 

3-4 

Offeror plans to complete a Life Cycle Cost analysis as part of the contract effort but did 
not describe the process that will be used. 

0-2 

Life Cycle Cost was not addressed in the Offeror’s proposal. 


Figure 19-7. Evaluation Factors Example 
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• Have life cycle support requirements been iden¬ 
tified (e.g., maintenance resource requirements, 
spare/repair parts, test and support equipment, 
personnel quantities and skills, etc?) Have these 
requirements been minimized to the extent 
possible through design? 

• Does the proposed design configuration reflect 
growth potential or change flexibility? 

• Has the supplier developed a comprehensive 
manufacturing and construction plan? Are key 
manufacturing processes identified along with 
their characteristics? 

• Does the supplier have an adequate quality 
assurance and statistical process control 
programs? 

• Does the supplier have a comprehensive 
planning effort (e.g., addresses program tasks, 
organizational structure and responsibilities, a 
WBS, task schedules, program monitoring and 
control procedures, etc.)? 


• Does the supplier’s proposal address all aspects 
of total life cycle cost? 

• Does the supplier have previous experience in 
the design, development, and production of 
system elements/components which are simi¬ 
lar in nature to the item proposed? 

Proposal Evaluation 

Proposal evaluation factors can be analyzed with 
any reasonable trade study approach. Figure 19-8 
shows a common approach. In this approach each 
factor is rated based on the evaluation factor ma¬ 
trix established for each criteria, such as that shown 
in Figure 19-7. It is then multiplied by a weight¬ 
ing factor based on the perceived priority of each 
criteria. All the weighted evaluations are added 
together and the highest score wins. 

Like trade studies the process should be examined 
for sensitivity problems; however, in the case of 
source selection, the check must be done with 
anticipated values prior to release of the RFP. 
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Evaluation Criteria 

WT. 

Factor 

(%) 

Proposal A 

Proposal B 

Proposal C 

Rating 

Score 

Rating 

Score 

Rating 

Score 

A. Technical Requirements: 

25 



■ 




1. Performance Characteristics 

6 

4 



30 



2. Effectiveness Factors 

4 

3 



16 



3. Design Approach 

3 

2 



9 

1 


4. Design Documentation 

4 

3 



16 


8 

5. Test and Evaluation Approach 

2 

2 



2 


4 

6. Product Support Requirements 

4 

2 

8 

H 

12 


8 

B. Production Capability 

20 







1. Production Layout 

8 

5 

40 

6 

48 

6 

48 

2. Manufacturing Process 

5 

2 

10 

3 

15 

4 

20 

3. Quality Control Assurance 

7 

5 

35 

6 

42 

4 

28 

C. Management 

20 







1. Planning (Plans/Schedules) 

6 

4 

24 

5 

30 

4 

24 

2. Organization Structure 

4 

4 

16 

4 

12 

4 

16 

3. Available Personnel Resources 

5 

3 

15 

3 

20 

3 

15 

4. Management Controls 

5 

3 

15 

3 

20 

4 

20 

D. Total Cost 

25 




■ 


■ 

1. Acquisition Price 

10 

7 

70 

5 


6 


2. Life Cycle Cost 

15 

9 

135 

10 

H 

8 


E. Additional Factors 

10 







1. Prior Experience 

4 

4 

16 

3 

12 

3 


2. Past Performance 

6 

5 

30 

5 

30 

3 

Bfl 

Grand Total 

100 


476 


* 

516 


450 

* Select Proposal B 


Figure 19-8. Source Evaluation 
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CHAPTER 20 


MANAGEMENT CONSIDERATIONS 

AND SUMMARY 


20.1 MANAGEMENT CONSIDERATIONS 

The Acquisition Reform Environment 

No one involved in systems acquisition, either 
within the department or as a supplier, can avoid 
considering how to manage acquisition in the 
current reform environment. In many ways, re¬ 
thinking the way we manage the systems engineer¬ 
ing process is implicit in reforming acquisition 
management. Using performance specifications 
(instead of detailed design specifications), leaving 
design decisions in the hands of contractors, 
delaying government control of configuration 
baselines—all are reform measures related directly 
to systems engineering management. This text has 
already addressed and acknowledged managing the 
technical effort in a reform environment. 

To a significant extent, the systems engineering 
processes—and systems engineers in general—are 
victims of their own successes in this environment. 
The systems engineering process was created and 
evolved to bring discipline to the business of pro¬ 
ducing very complex systems. It is intended to 
ensure that requirements are carefully analyzed, 
and that they flow down to detailed designs. The 
process demands that details are understood and 
managed. And the process has been successful. 
Since the 1960s manufacturers, in concert with 
government program offices, have produced a 
series of ever-increasingly capable and reliable 
systems using the processes described in this text. 
The problem is, in too many cases, we have over¬ 
laid the process with ever-increasing levels of 
controls, reports, and reviews. The result is that 
the cycle time required to produce systems has 
increased to unacceptable levels, even as technol¬ 
ogy life cycles have decreased precipitously. The 


fact is that, in too many cases, we are producing 
excellent systems, but systems that take too long 
to produce, cost too much, and are often outdated 
when they are finally produced. The demand for 
change has been sounded, and systems engineer¬ 
ing management must respond if change is to take 
place. The question then becomes how should one 
manage to be successful in this environment? We 
have a process that produces good systems; how 
should we change the process that has served us 
well so that it serves us better? 

At the heart of acquisition reform is this idea: we 
can improve our ability to provide our users with 
highly capable systems at reasonable cost and 
schedule. We can if we manage design and devel¬ 
opment in a way that takes full advantage of the 
expertise resident both with the government and 
the contractor. This translates into the government 
stating its needs in terms of performance outcomes 
desired, rather than in terms of specific design 
solutions required; and, likewise, in having con¬ 
tractors select detailed design approaches that 
deliver the performance demanded, and then 
taking responsibility for the performance actually 
achieved. 

This approach has been implemented in DoD, and 
in other government agencies as well. In its earlier 
implementations, several cases occurred where the 
government managers, in an attempt to ensure that 
the government did not impose design solutions 
on contractors, chose to deliberately distance the 
government technical staff from contractors. This 
presumed that the contractor would step forward 
to ensure that necessary engineering disciplines and 
functions were covered. In more than one case, 
the evidence after the fact was that, as the 
government stepped back to a less directive role 
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in design and development, the contractor did not 
take a corresponding step forward to ensure that 
normal engineering management disciplines were 
included. In several cases where problems arose, 
after-the-fact investigation showed important ele¬ 
ments of the systems engineering process were 
either deliberately ignored or overlooked. 

The problem in each case seems to have been 
failure to communicate expectations between the 
government and the contractor, compounded by a 
failure on the part of the government to ensure that 
normal engineering management disciplines were 
exercised. One of the more important lessons 
learned has been that while the systems engineer¬ 
ing process can—and should be—tailored to the 
specific needs of the program, there is substantial 
risk ignoring elements of the process. Before one 
decides to skip phases, eliminate reviews, or take 
other actions that appear to deliver shortened 
schedules and less cost, one must ensure that 
those decisions are appropriate for the risks that 
characterize the program. 

Arbitrary engineering management decisions yield 
poor technical results. One of the primary require¬ 
ments inherent in systems engineering is to assess 
the engineering management program for its con¬ 
sistency with the technical realities and risks con¬ 
fronted, and to communicate his/her findings and 
recommendations to management. DoD policy is 
quite clear on this issue. The government is not, in 
most cases, expected to take the lead in the devel¬ 
opment of design solutions. That, however, does 
not relieve the government of its responsibilities 
to the taxpayers to ensure that sound technical and 
management processes are in place. The systems 
engineer must take the lead role in establishing the 
technical management requirements for the pro¬ 
gram and seeing that those requirements are com¬ 
municated clearly to program managers and to the 
contractor. 

Communication - Trust and Integrity 

Clearly, one of the fundamental requirements for 
an effective systems engineer is the ability to com¬ 
municate. Key to effective communication is the 


rudimentary understanding that communication 
involves two elements—a transmitter and a 
receiver. Even if we have a valid message and the 
capacity for expressing our positions in terms that 
enable others to understand what we are saying, 
true communication may not take place if the 
intended receiver chooses not to receive our mes¬ 
sage. What can we do, as engineering managers to 
help our own cause as far as ensuring that our 
communications are received and understood? 

Much can be done to condition others to listen and 
give serious consideration to what one says, and, 
of course, the opposite is equally true—one can 
condition others to ignore what he/she says. It is 
primarily a matter of establishing credibility based 
on integrity and trust. 

First, however, it is appropriate to discuss the 
systems engineer’s role as a member of the man¬ 
agement team. Systems engineering, as practiced 
in DoD, is fundamentally the practice of engineer¬ 
ing management. The systems engineer is expected 
to integrate not only the technical disciplines in 
reaching recommendations, but also to integrate 
traditional management concerns such as cost, 
schedule, and policy into the technical manage¬ 
ment equation. In this role, senior levels of man¬ 
agement expect the systems engineer to understand 
the policies that govern the program, and to ap¬ 
preciate the imperatives of cost and schedule. Fur¬ 
thermore, in the absence of compelling reasons to 
the contrary, they expect support of the policies 
enunciated and they expect the senior engineer to 
balance technical performance objectives with cost 
and schedule constraints. 

Does this mean that the engineer should place his 
obligation to be a supportive team member above 
his ethical obligation to provide honest engineer¬ 
ing judgment? Absolutely not! But it does mean 
that, if one is to gain a fair healing for expression 
of reservations based on engineering judgment, one 
must be viewed as a member of the team. The indi¬ 
vidual who always fights the system, always ob¬ 
jects to established policy, and, in general, refuses 
to try to see other points of view will eventually 
become isolated. When others cease listening, the 
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communication stops and even valid points of view 
are lost because the intended audience is no longer 
receiving the message—valid or not. 

In addition to being team players, engineering 
managers can further condition others to be recep¬ 
tive to their views by establishing a reputation for 
making reasoned judgments. A primary require¬ 
ment for establishing such a reputation is that man¬ 
agers must have technical expertise. They must be 
able to make technical judgments grounded in a 
sound understanding of the principles that govern 
science and technology. Systems engineers must 
have the education and the experience that justi¬ 
fies confidence in their technical judgments. In the 
absence of that kind of expertise, it is unlikely that 
engineering managers will be able to gain the re¬ 
spect of those with whom they must work. And 
yet, systems engineers cannot be expert in all the 
areas that must be integrated in order to create a 
successful system. Consequently, systems engi¬ 
neers must recognize the limits of their expertise 
and seek advice when those limits are reached. 
And, of course, systems engineers must have built 
a reputation for integrity. They must have demon¬ 
strated a willingness to make the principled stand 
when that is required and to make the tough call, 
even when there are substantial pressures to do 
otherwise. 

Another, perhaps small way, that engineers can 
improve communication with other members of 
their teams (especially those without an engineer¬ 
ing background) is to have confidence in the posi¬ 
tion being articulated and to articulate the position 
concisely. The natural tendency of many engineers 
is to put forward their position on a subject along 
with all the facts, figures, data and required proofs 
that resulted in the position being taken. This some¬ 
times results in explaining how a watch works 
when all that was asked was “What time is it?” 
Unless demonstrated otherwise, team members 
will generally trust the engineer’s judgment and 
will assume that all the required rationale is in 
place, without having to see it. There are some 
times when it is appropriate to describe how the 


watch works, but many times communication is 
enhanced and time saved by providing a confident 
and concise answer. 

When systems engineers show themselves to be 
strong and knowledgeable, able to operate effec¬ 
tively in a team environment, then communication 
problems are unlikely to stand in the way of effec¬ 
tive engineering management. 

20.2 ETHICAL CONSIDERATIONS 

The practice of engineering exists in an environ¬ 
ment of many competing interests. Cost and sched¬ 
ule pressures; changes in operational threats, 
requirements, technology, laws, and policies; and 
changes in the emphasis on tailoring policies in a 
common-sense way are a few examples. These 
competing interests are exposed on a daily basis 
as organizations embrace the integrated product 
and process development approach. The commu¬ 
nication techniques described earlier in this chap¬ 
ter, and the systems engineering tools described in 
earlier chapters of this book, provide guidance for 
engineers in effectively advocating the importance 
of the technical aspects of the product in this envi¬ 
ronment of competing interests. 

But, what do engineers do when, in their opinion, 
the integrated team or its leadership are not put¬ 
ting adequate emphasis on the technical issues? 
This question becomes especially difficult in the 
cases of product safety or when human life is at 
stake. There is no explicit set of rules that directs 
the individual in handling issues of ethical integ¬ 
rity. Ethics is the responsibility of everyone on the 
integrated team. Engineers, while clearly the ad¬ 
vocate for the technical aspects of theintgrated 
solution, do not have a special role as ethical 
watchdogs because of their technical knowledge. 

Richard T. De George in his article entitled Ethical 
Responsibilities of Engineers in Large Organiza¬ 
tions: The Pinto Case 1 makes the following case: 
“The myth that ethics has no place in engineering 


1 Ethical Issues in Engineering, Johnson, Ch 15. 
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has been attacked, and at least in some comers of 
the engineering profession been put to rest. Another 
myth, however, is emerging to take its place—the 
myth of the engineer as moral hero.” 

This emphasis, De George believes, is misplaced. 
“The zeal of some preachers, however, has gone 
too far, piling moral responsibility upon moral re¬ 
sponsibility on the shoulders of the engineer. 
Though engineers are members of a profession that 
holds public safety paramount, we cannot reason¬ 
ably expect engineers to be willing to sacrifice their 
jobs each day for principle and to have a whistle 
ever by their sides ready to blow if their firm strays 
from what they perceive to be the morally right 
course of action.” 

What then is the responsibility of engineers to 
speak out? De George suggests as a rule of thumb 
that engineers and others in a large organization 
are morally permitted to go public with informa¬ 
tion about the safety of a product if the following 
conditions are met: 

1. If the harm that will be done by the product to 
the public is serious and considerable. 

2. If they make their concerns known to their 
superiors. 

3. If, getting no satisfaction from their immedi¬ 
ate supervisors, they exhaust the channels 
available within the operation, including going 
to the board of directors (or equivalent). 

De George believes if they still get no action at 
this point, engineers or others are morally permit¬ 
ted to make their concerns public but not morally 
obligated to do so. To have a moral obligation to 
go public he adds two additional conditions to those 
above: 

4. The person must have documented evidence 
that would convince a reasonable, impartial 
observer that his/her view of the situation is 
correct and the company policy wrong. 


5. There must be strong evidence that making the 
information public will in fact prevent the 
threatened serious harm. 

Most ethical dilemmas in engineering management 
can be traced to different objectives and expecta¬ 
tions in the vertical chain of command. Higher 
authority knows the external pressures that impact 
programs and tends to focus on them. System 
engineers know the realities of the on-going 
development process and tend to focus on the 
internal technical process. Unless there is commu¬ 
nication between the two, misunderstandings and 
late information can generate reactive decisions and 
potential ethical dilemmas. The challenge for sys¬ 
tem engineers is to improve communication to help 
unify objectives and expectations. Divisive ethi¬ 
cal issues can be avoided where communication is 
respected and maintained. 

20.3 SUMMARY 

The material presented in this book is focused on 
the details of the classic systems engineering 
process and the role of the systems engineer as the 
primary practitioner where the activities included 
in that process are concerned. The systems engi¬ 
neering process described has been used success¬ 
fully in both DoD and commercial product devel¬ 
opment for decades. In that sense, little new or revo¬ 
lutionary material has been introduced in this text. 
Rather, we have tried to describe this time-proven 
process at a level of detail that makes it logical 
and understandable as a tool to use to plan, design, 
and develop products that must meet a defined set 
of requirements. 

In DoD, systems engineers must assume roles of 
engineering managers on the program or project 
assigned. They must understand that the role of 
the systems engineer is necessarily different from 
that normal to the narrowly specialized functional 
engineer, yet it is also different from the role played 
by the program manager. In a sense, the role of the 
systems engineer is a delicate one, striving to bal¬ 
ance technical concerns with the real management 
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pressures deriving from cost, schedule, and policy. 
The systems engineer is often the person in the 
middle; it is seldom a comfortable position. This 
text has been aimed at that individual. 

The first two parts of the text were intended to first 
give the reader a comprehensive overview of sys¬ 
tems engineering as a practice and to demonstrate 
the role that systems engineering plays within the 
DoD acquisition management process. Part 2, in 
particular, was intended to provide relatively de¬ 
tailed insights into the specific activities that make 
up the process. The government systems engineer 
may find him/herself deeply involved in some of 
the detailed activities that are included in the pro¬ 
cess, while less involved in others. For example, 
government systems engineers may find them¬ 
selves very involved in requirements definition and 
analysis, but less directly involved in design syn¬ 
thesis. Flowever, the fact that government engineers 
do not directly synthesize designs does not relieve 
them from a responsibility to understand the 
process and to ensure that sound practices are 
pursued in reaching design decisions. It is for this 
reason that understanding details of the process 
are critical. 

Part 3 of the book is perhaps the heart of the text 
from an engineering management perspective. In 
Part 3, we have presented discussions on a series 
of topics under the general heading of Systems 
Analysis and Control. The engine that translates 
requirements into designs is defined by the require¬ 
ments analysis, functional analysis and allocation, 
and design synthesis sequence of activities. Much 


of the role of the systems engineer is to evaluate 
progress, consider alternatives, and ensure the prod¬ 
uct remains consistent and true to the requirements 
upon which the design is based. The tools and tech¬ 
niques presented in Part 3 are the primary means 
by which a good engineering management effort 
accomplishes these tasks. 

Finally, in Part 4, we presented some of the 
considerations beyond the implementation of a 
disciplined systems engineering process that the 
engineering manager must consider in order to be 
successful. Particularly in today’s environment 
where new starts are few and resources often lim¬ 
ited, the planning function and the issues associ¬ 
ated with product improvement and integrated team 
management must move to the forefront of the 
systems engineer’s thinking from the very early 
stages of work on any system. 

This book has attempted to su mm arize the primary 
activities and issues associated with the conduct 
and management of technical activities on DoD 
programs and projects. It was written to supple¬ 
ment the material presented courses at the Defense 
Systems Management College. The disciplined 
application of the principles associated with 
systems engineering has been recognized as one 
indicator of likely success in complex programs. 
As always, however, the key is for the practitioner 
to be able to absorb these fundamental principles 
and then to tailor them to the specific circumstances 
confronted. We hope that the book will prove use¬ 
ful in the future challenges that readers will face 
as engineering managers. 
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GLOSSARY 

SYSTEMS ENGINEERING 
FUNDAMENTALS 

AAAV Advanced Amphibious Assault Vehicle 
ACAT Acquisition Category 
ACR Alternative Concept Review 
AMSDL Acquisition Management Systems Data List 
ASR Alternative Systems Review 
AUPP Average Unit Procurement Price 
AWP Awaiting Parts 

BL Baseline 

BLRIP Beyond Low Rate Initial Production 

C4ISR Command, ontrol, Communications, Computers, Intelligence, 
and Reconnaissance 

CAD Computer-Aided Design 

CAE Computer-Aided Engineering 

CAIV Cost As an Independent Variable 

CALS Continuous Acquisition and Life Cycle Support 

CAM Computer-Aided Manufacturing 

CASE Computer-Aided Systems Engineering 

CATIA Computer-Aided Three-Dimensional Interactive Application 

CCB Configuration Control Board 

CCR Contract Change Request 

CDR Critical Design Review 

CDRL Contract Data Requirement List 

CDS Concept Design Sheet 

CE Concept Exploration 
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CEO 

Cl 

Circular A-109 
CM 
CM 
COTS 
CSCI 
CWI 


Chief Executive Officer 
Configuration Item 
Major Systems Acquisitions 
Configuration Management 
Control Manager 
Commercial Off-The-Shelf 
Computer Software Configuration Item 
Continuous Wave Illumination 


DAU 

DCMC 

DDR 

DFARS 

DID 

DoD 

DoD 5000.2-R 

DoDISS 

DSMC 

DT 

DTC 

DT&E 


Defense Acquisition University 
Defense Contract Management Command 
Detail Design Review 

Defense Supplement to the Federal Acquisition Regulation 
Data Item Description 
Department of Defense 

Mandatory Procedures for Major Defense Acquisition Programs (MDAPs), and 
Major Automated Information System Acquisition Programs (MAIS) 

DoD Index of Specifications and Standards 

Defense Systems Management College 

Developmental Testing 

Design To Cost 

Developmental Test and Evaluation 


EC 

ECP 

EDI 

EIA 
EIA IS 632 
EIA IS-649 


Engineering Change 
Engineering Change Proposal 
Electronic Data Interchange 

Electronic Industries Alliance 

Electronic Industries Association Interim Standard 632, on Systems Engineering 

Electronic Industries Association Interim Standard 649, on Configuration 
Management 


EOA Early Operational Assessments 
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FAR Federal Acquisition Regulation 
FCA Functional Configuration Audit 
FEO Field Engineering Order 
FFBD Functional Flow Block Diagram 
FIPS Federal Information Processing Standard 
FMECA Failure Modes, Effects, and Criticality Analysis 
FOT&E Follow-On Operational Test and Evaluation 
FQR Formal Qualification Review 

GFE Government Furnished Equipment 
GFM Government Furnished Material 


ICD 

ICWG 

IDE 

IDEF 

IDEFO 

IDEFlx 

IEEE 

IEEE/EIA 12207 
IEEE P1220 

IFB 

IIPT 

IMS 

IOC 

IOT&E 

IPPD 

IPR 

IPT 


Interface Control Documentation 

Interface Control Working Group 

Integrated Digital Environment 

Integration Definition Function 

Integrated Definition for Function Modeling 

Integration Definition for Information Modeling 

Institute of Electrical and Electronics Engineers 

IEEE/EIA Standard 12207, Software Life Cycle Processes 

IEEE Draft Standard 1220, Application and Management of the Systems 
Engineering Process 

Invitation for Bid 

Integrating Integrated Product Teams 

Integrated Master Schedule 

Initial Operational Capability 

Initial Operational Test and Evaluation 

Integrated Product and Process Development 

In-Progress/Process Review 

Integrated Product Teams 
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JASSM Joint Air-to-Surface Standoff Missile 
JROC Joint Requirements Oversight Council 
JTA Joint Technical Architecture 

KPPs Key Performance Parameters 

LFT&E Live Fire Test and Evaluation 
LRU Line-Replaceable Unit 
LRIP Low Rate Initial Production 


M&S 
MAIS 
MAISRC 
MBTF 
MDA 
MDAP 
MIL-HDBK-61 
MIL-HDBK-881 
MIL-STD 499A 
MIL-STD-961D 
MIL-STD 962 
MIL-STD-973 
MNS 
MOE 
MOP 
MOS 
MRP II 
MS 
MTTR 


Modeling and Stimulation 

Major Automated Information System 

Major Automated Information Systems Review Council 

Mean Time Between Failure 

Milestone Decision Authority 

Major Defense Acquisition Program 

Military Handbook 61, on Configuration Management 

Military Handbook 881, on Work Breakdown Structure 

Military Standard 499A, on Engineering Management 

Military Standard 961D, on Standard Practice for Defense Specifications 

Military Standard 962, on Format and Content of Defense Standards 

Military Standard 973, on Configuration Management 

Mission Need Statement 

Measure of Effectiveness 

Measure of Performance 

Measure of Suitability 

Manufacturing Resource Planning II 

Milestone 

Mean Time To Repair 


NDI Non-Developmental Item 
NIST National Institute of Standards and Technology 
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NRTS Not Repairable This Station 

OA Operational Assessment 
OIPT Overarching Integrated Product Teams 
OMB Office of Management and Budget 
OPS Operations 

ORD Operational Requirements Document 
OSD Office of the Secretary of Defense 
OT&E Operational Test and Evaluation 

P3I Preplanned Product Improvement 
PAR Production Approval Reviews 
PCA Physical Configuration Audit 
PDR Preliminary Design Review 
PDRR Program Definition and Risk Reduction 
PEO Program Executive Office 
PM Program Manager 
PME Program/Project Manager - Electronics 
PMO Program Management Office 
PMT Program Management Team 
PPBS Planning, Programming and Budgeting System 
PRR Production Readiness Review 

QA Quality Assurance 
QFD Quality Function Deployment 

R&D Research and Development 
RAS Requirements Allocation Sheets 
RCS Radar Cross Section 

RDT&E Research, Development, Test and Evaluation 
RFP Request for Proposal 
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S&T 

SBA 

SBD 

SD&E 

SDefR 

SDR 

SE 

Section L 

Section M 
SEDS 
SEMS 
SEP 
SFR 
SI 
SI&T 
SOO 
SOW 
SPEC 
SSA 
SSAC 
SSEB 
SSP 
SSR 
SRR 
SRU 
STD 
SVR 

s/w 


Science and Technology 

Simulation Based Acquisition 

Schematic Block Diagram 

System Development and Demonstration 

System Definition Review (as referred to in IEEE PI220) 

System Design Review 

Systems Engineering 

Instructions to Offerors (Portion of Uniform Contract Format) 

Evaluation Criteria (Portion of Uniform Contract Format) 

Systems Engineering Detail Schedule 

Systems Engineering Master Schedule 

Systems Engineering Process 

System Functional Review 

Software Item 

System Integration and Test 

Statement of Objectives 

Statement of Work 

Specification 

Source Selection Authority 
Source Selection Advisory Council 
Source Selection Evaluation Board 
Source Selection Plan 
Software Specification Review 
System Requirements Review 
Shop-Replaceable Unit 
Standard 

System Verification Review 
Software 


214 



Glossary 


Systems Engineering Fundamentals 


T&E Test and Evaluation 
TDP Technical Data Package 
TEMP Test and Evaluation Master Plan 
TLS Timeline Analysis Sheet 
TOC Team Operating Contract 
TPM Technical Performance Measurement 
TPWG Test Planning Work Group 
TRR Test Readiness Review 

VV&A Verfication, Validation, and Accreditation 

WIPT Working-Level Integrated Product Team 
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